FINITE TRACES AND REPRESENTATIONS OF THE GROUP 
OF INFINITE MATRICES OVER A FINITE FIELD. 



VADIM GORIN, SERGEI KEROV, AND ANATOLY VERSHIK 



Abstract. The article is devoted to the representation theory of locally com- 
pact group GLB of almost upper-triangular infinite matrices over finite field, 
which we view as an adequate n = oo analogue of general linear groups 
GL(n, q). We describe all semifinite traces (characters) of GLB which are 
finite on the appropriate subspace _4(GLB) of smooth functions in L\ on the 
group. 

We further distinguish a class of unipotent traces and explore their proper- 
ties, including remarkable multiplicativity and connections with conjugation- 
invariant probability measures on the group of upper-triangular matrices over 
finite field. We construct representations of GLB corresponding to the wide 
class of indecomposable unipotent traces. Our construction is based on the nat- 
ural action of GLB in the space of flags in the countable infinite-dimensional 
vector space and leads to von Neumann type /Too factor representations. 

We also study and decompose the (bi-)regular representation of GLB. Fi- 
nally, various connections between representation theory of GLB and rep- 
resentation theory of the infinite-dimensional Iwahori-Hecke algebra Hq(oo) 
and infinite symmetric group S(oo) (which is the inductive limit of symmetric 
groups S(n)) are explained. 

The main stream of this paper is a continuation and development of the 
series of the previous papers by S.Kerov and A.Vershik of 70-90s on the repre- 
sentation theory of finite and infinite symmetric groups. The topics we study 
should be considered, in general, as a part of the so-called asymptotic repre- 
sentation theory. 



Contents 

Historical preface 2 

1. Introduction 3 

1.1. Overview 3 

1.2. Shwartz-Bruhat algebra „4(GLB) and its traces 4 

1.3. Unipotent traces 5 

1.4. Construction of representations of GLB 7 

1.5. Biregular representation of GLB. 9 

1.6. Motivations and comments 9 

1.7. List of main notations and theorems 11 

1.8. Acknowledgements 12 

2. The group GLB and its Schwartz-Bruhat algebra _4(GLB) 12 

2.1. Basic definitions 12 

2.2. A(GLB) as an inductive limit 13 

2.3. Facts from representation theory of GL(n, q) [14 



MSC 2010: 22D10, 22D25, 20G40 



2 



VADIM GORIN, SERGEI KEROV, AND ANATOLY VERSHIK 



2.4. Structure of _4(GLB) 

2.5. Some subalgebras of „4(GLB) 

2.6. Classification of traces of „4(GLB) 

3. Unipotent traces and their values 

3.1. Values of unipotent traces: formulations 

3.2. Values of unipotent traces: proofs 

3.3. Restriction of unipotent traces to Iwahori-Hecke algebra 

4. Identification of unipotent traces with probability measures 

5. Gruppoid construction for the representations of GLB 

5.1. Generalities 

5.2. Two simplest type / examples 

5.3. Representations related to grassmanian 

5.4. Representations related to spaces of flags 

6. Biregular representation of GLB 

7. Appendix: GLU 
References 



Historical preface 

My joint work with S. Kerov on the asymptotic representation theory of the 
matrix groups GL(n, q) over finite field as the rank n grows to infinity, was started at 
the beginning of 80s as a continuation of our papers devoted to analogous problems 
for symmetric groups of growing ranks. The problem setup was taking into account 
not only the analogy between these two series of groups, but also an important 
distinction in the definition of the inductive limit of group algebras, which should 
be modified for <&h(n,q). "True" (i.e. parabolic) embedding of these group algebras 
was well-known starting from the very first papers on the representation theory of 
GL(n, q) (see |Gr| . |Zel| . |F]. etc). It was used by A. Zelevinsky and us (see |V82] ) 
to define a natural limit object (i.e. inductive limit) which is the group GLB of 
infinite matrices (over finite field) with finitely many non-zero elements below the 
main diagonal. The results of the book |Zel) in which the representation theory 
of GL(n, q) is studied via the Hopf algebras theory, are not directly related to the 
asymptotic representation theory but are substantial for the problems' setup. 

However, the project was suspended and we returned to this topic only in the 
middle of 90s with the first article |VK98] appearing in 1998. In this paper we 
gave main definitions and sketched the plan of further research. In 1998-1999 we 
prepared some more detailed texts. Already after the sad death of my former stu- 
dent and coauthor an outstanding mathematician and person S. Kerov (1946-2000) 
an improved version of these texts was published |VK07| . Also lecture notes |V03j 
based on my talks at school EMS-NATO in The Euler International Mathematical 
Institute in 2001 were published a bit earlier (2003). Several talks devoted to this 
topic were presented at various conferences and seminars. All these texts contained 
(mostly without proofs) a number of statements forming an initial foundation of 
the asymptotic representation theory of group GLB. 

V. Gorin following the general plan contained in |VK07| interpreted and supple- 
mented with complete proofs the statements of that article. The present paper is 
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a result of this work. We can say that this article, finally, concludes the first step 
in the study of the representation theory of GLB. 

Further work related to this important topic will, probably, be based on the 
scheme presented here and might have several directions. First, a more detailed 
study of the principal representations (i.e. representations induced from the Borel 
subgroup) is required; this part of theory is especially close to the representation 
theory of the infinite symmetric group. Second, we need to find a more explicit 
reduction of other representations, including cuspidal ones and others, to the prin- 
cipal representations. The most interesting questions are related to the harmonic 
analysis and combinatorics of the group algebra of GLB, to the interpretation of 
characters as measures, to statistics of Jordan normal forms, etc, and also to the 
representations of this group, both irreducible and factor representations. One can 
hope that this theory will be of use for the general representation theory of locally 
compact groups and will also extend our understanding of the representation theory 
of groups GL(n, q). 

A. Vershik 

1. Introduction 

1.1. Overview. Let ¥ q be the finite field with q elements and let GL(n, q) denote 
the group of all invertible n x n matrices over ¥ q . In the present article we study the 
group GLB of all almost upper-triangular matrices (i.e. containing finitely many 
nonzero elements below diagonal) over ¥ q , which we view as n — oo analogue of 
groups Gh(n,q), as will be explained in Section 11.61 In other words, if Voo is the 
(countable) vector space of all finite vectors in ¥ q a and V n is the subspace spanned 
by the first n basis vectors, then GLB consists of all linear transforms of 
preserving all but finitely many spaces V n . 

An important feature of the infinite-dimensional group GLB is its local compact- 
ness. GLB possesses the Haar measure /iglb which allows to introduce the group 
algebra ii(GLB, /iglb) with multiplication given by the convolution. In our study 
we intensively use an important dense subalgebra ^l(GLB) of ii(GLB, ^glb) con- 
sisting of all continuous functions with compact support taking only finitely many 
values. 

As we show the group GLB has a rich family of traces (characters) , which might 
be singular on the group itself, but are finite on „4(GLB). These traces give rise to 
type II factor representations and form a basis for the representation theory and 
harmonic analysis on GLB. They are the main object of our interest. 

In the present article we concentrate on the following topics, where we prove a 
variety of results. 

• We study the algebra Li(GLB, /jglb) and its dense locally semisimple 
subalgebra _4(GLB). The structure of the latter as an inductive limit of 
finite-dimensional algebras is explained and thoroughly investigated. 

• We describe all finite traces of _4(GLB). 

• We distinguish the family of unipotent (principal) traces, find their re- 
markable properties and explain their connections with infinite-dimensional 
Iwahori-Hecke algebra H q (oo), infinite symmetric group S(oo) and unipo- 
tent representations of GL(n, q). 

• We show that each unipotent trace can be identified with conjugation- 
invariant ergodic probability measure on the Borel subgroup B C GLB of 
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all upper-triangular matrices. A number of theorems and conjectures on 
the structure of such measures is presented. 

• We give a construction for the wide family of representations of GLB 
based on the natural action of GLB in the spaces of flags in the infinite- 
dimensional vector space over ¥ q and in the principal gruppoid defined by 
this action. We prove that these representations are von Neumann type 
I loo factor representations and compute their traces, which are identified 
with extreme (indecomposable) unipotent traces of _4(GLB) with explicit 
parameters in our classification of all finite traces. 

• We study the (bi-)-regular representation of GLB, show that is possesses 
a natural trace which is finite on _4(GLB) and decompose this trace into 
extremes. 

In Sections I1.2H1.5I a more detailed description of our work is given. In Section 
11.61 we motivate our definitions and some of the choices, which otherwise might 
seem arbitrary. We also summarize the similarities with asymptotic representation 
theory of symmetric groups in the same section. The brief list of key notations 
and theorems is given in Section 11.71 Finally, we want to remark that most of the 
results of the present paper were announced (without proofs) in articles |VK98| . 
|V03j . [VK07| . In this text we refine those statements and give full proofs of them, 
also a number of entirely new results is presented. 

1.2. Shwartz-Bruhat algebra „4(GLB) and its traces. One of the central ob- 
jects of our study is algebra .A(GLB) C Li(GLB, /iglb) consisting of all con- 
tinuous functions on GLB with compact support taking only finitely many val- 
ues. ^l(GLB) can be viewed as an analogue of the algebra of smooth functions or 
Shwartz-Bruhat algebra in the theory of linear p-adic groups, see e.g. |BZj . |GGP| . 

In Section[2]we show that algebra .A(GLB) is an inductive limit of group algebras 
■A(GLB)„ ~ C(GL(n,q))). However, the arising embeddings i n : C(GL(n,q)) e -t 
C(GL(n + l,q)) are not induced by group embeddings, instead we should use par- 
abolic emebeddings, which are averagings by certain subgroups. This description 
implies that .A(GLB) is locally semisimple algebra, which means, in particular, 
that the enveloping C*-algebra of GLB is almost finite dimensional (AF-) algebra; 
see e.g. |EJ, |VK87| . [SV] . [K03] , 

As for every locally semisimple algebra the structure of the algebra „4(GLB) is 
uniquely defined by its Bratteli diagram. Recall that Bratteli diagram is a graded 
graph with vertices at level n symbolizing irreducible representations of _4.(GLB)„ 
and edges between adjacent levels symbolizing the inclusion relations of the repre- 
sentations. The inclusions i n : „4(GLB) n „4(GLB) n+ i are not unital and the 
algebra _4(GLB) has no unit element, thus, each vertex of the Bratteli diagram 
is to be supplemented with additional label which is the dimension of the corre- 
sponding irreducible representation. In our case these numbers are dimensions of 
irreducible representations of groups GL(n, q) and they admit explicit formulas (e.g. 
a g-analogue of the classical hook formula, see e.g. [Ml Chapter IV]) 

We show in Section [2] that the Bratteli diagram of .A(GLB) is a union of count- 
ably many copies of the Young graph Y with shifted grading. Recall that level 
n of Y consist of all Young diagrams with n boxes (equivalently, partitions of n) 
with edges joining the diagrams which differ by addition of a single box. Therefore, 
the algebra ^l(GLB) is a direct sum of countably many (non-isomorphic) ideals 
corresponding to different copies of Y. 
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We also show that infinite Iwahori-Hecke algebra TL q (oo) is naturally embedded 
into „4(GLB), moreover, it is a subset of one of the above ideals. This somehow 
explains the appearance of the Young graph. Indeed, Y (without any labels, since 
T-L q {oo) contains the unit element) is the Bratteli diagram of T-L q (oo) as follows from 
the fact that H q (oo) and the group algebra of 5(oo) are isomorphic, see e.g. |VK89| . 

We further concentrate on the representation theory of .A(GLB) and GLB. It 
is well-known that the representation theory of "big" groups, such as S'(oo), U(oo), 
GIL (00,(7) is wild and in order to get a well-behaved theory one has to restrict 
the class of the representations under consideration. There are two approaches 
here, one of them deals with von Neumann factor representations, see e.g. |Th84| . 
and another one studies representations of (G, K )-pairs, see e.g. [02J. In both 
approaches representations are in correspondence with characters (or traces) of a 
group (or algebra), thus, it is crucial to obtain the classification of traces. 

In Section T2.6I we describe the set of finite traces of A(GLB). We show (see 
Theorem I2.26|) that this set is a simplicial cone with extreme rays parameterized 
by triplets (f,a,0), where a = {ai} and j3 — {Pi} are non-increasing sequences of 
non-negative reals satisfying: 

00 

ai >a 2 > •••>0, Pi >i3 2 > ••■>0, ^(oi + ft)<l, 

i=l 

and / is an element of a certain (explicit) countable set CY'. This result is to a large 
extent based on the classification theorem for characters of the infinite symmetric 
group S(oo) first proved by Thoma in |Th64| . For each extreme trace parameterized 
by (f,a,/3) we give a formula for its values on .A(GLB) in terms of the values of 
characters of irreducible representations of GL(n, q). 

1.3. Unipotent traces. Among the traces of .A(GLB) we distinguish a class of 
unipotent traces, which are closely related to the same-named representations of 
GL(n, q). In our classification of traces the unipotent extreme ones are such that 
f = fo, where fo is a certain special element of CY'. They are distinguished by 
the fact that the values on (at least some) elements of W q (oo) C -4(GLB) are 
non-zero. On the contrary, if / ^ /o, then for arbitrary a and (3 the corresponding 
trace vanishes on T-L q {oo). Moreover, the restriction of extreme unipotent trace on 
_4.(GLB)„ ~ C(GL(n, (7)) is a linear combination of traces of irreducible unipo- 
tent representations of GL(n, q), see (St], |J2| for more information on unipotent 
representation of GL(n, q). 

The extreme unipotent traces are in one-to-one correspondence with extreme 
traces of Iwahori-Hecke algebra H q (oo) (and, thus, with characters of the infinite 
symmetric group S'(oo)). Extreme traces of Ji q (oo) are parameterized (see |VK89| 
and also [Me, Section 7]) by two sequences 

01 >a 2 >••■>, 0i >Aa>"->0, ^(a i +A)<l 

i 

and the (normalized) restriction of the extreme unipotent trace of .A(GLB) on 
H q (oo) C .A(GLB) is extreme trace of T-L q {oo) with the same parameters. This pro- 
vides an infinite-dimensional analogue of the well-known correspondence between 
irreducible representations of H q (n) and unipotent irreducible representations of 
GL(n,q), see e.g. [CF]. 
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The extreme unipotent traces have a number of intriguing properties which we 
discuss in Section [3] 

Recall the following multiplicativity property for the characters of S(oo), see 
|Th64| . |VK81| . |KOO| . Let y be a normalized character (i.e. central positive- 
definite function satisfying x(e) = 1) of S(oo). Then \ i s extreme (i.e. extreme 
point of the convex set of all normalized characters) if and only if the following 
multiplicativity property is satisfied. For g g S(n) C S(oo) and h £ S(m) C S(oo) 
let g h denote the element of S(n + m) C S'(oo) obtained by adjoining the 
permutations g and h, i.e. by making g act on 1, . . . , n and h act on n+1, ... ,n + m. 
Then for all n,m, g,h we have 

(i-i) x(g®h)= x (g)x(h). 

We prove (see Theorem I3.3[) that extreme unipotent traces of „4(GLB) satisfy 
an analogue of (jl.ip . More precisely, let e g denote the element of ^(GLB)„ C 
.A(GLB) corresponding to g under the identification of .A(GLB)„ and group al- 
gebra of Gh(n,q). Further, for g 6 GL(n, q), h € GL(m, q) let g h denote the 
block-diagonal matrix in GrL,(n + m, q) with blocks n and m. Then every normalized 
extreme unipotent trace \ of »4(GLB) satisfies 

x(egQh) = x(e g )x(eh)- 

for any g, h with coprime characteristic polynomials. We further compute the 
values of extreme unipotent traces on arbitrary elements of ^l(GLB) in terms of 
explicit specializations (i.e. homomorphisms into C) of the algebra of symmetric 
functions A, see Theorem 13.51 

We continue the discussion of unipotent traces by showing their relation to cer- 
tain probability measures. We prove (see Theorem l4.ip that each extreme unipotent 
trace of ^l(GLB) gives rise to a probability measure on the subgroup B C GLB of 
upper-triangular matrices and, moreover, if two traces are not proportional, then 
corresponding measures are distinct. Also each such measure on B can be natu- 
rally extended to a signed (i.e. not necessary positive) cr-finite measure on GLB 
invariant under conjugations. This gives an interpretation of traces of ^l(GLB) as 
characters of the group GLB which are infinite (i.e. not well-defined) on the group 
itself, but have a singularity of type measure. 

We further study the properties of measures on B corresponding to the unipotent 
traces. We show (see Theorem l4.1|) that each such measure is ergodic (with respect 
to conjugations). Motivated by this connection we turn our attention to study of 
ergodic probability measures on B. 

At this point in order to simplify the exposition it is convenient to switch from 
GLB to another distinguished infinite-dimensional matrix group GLU. GLU is 
the group group of all almost uni-uppertriangular infinite matrices, i.e. 

GLU = {[Xij] e GLB : X tl = 1 for large enough i}. 

The whole theory for GLU is very much parallel to that of GLB, and we summarize 
it in the Appendix. For now we only need the fact that GLU also has a distin- 
guished class of extreme unipotent traces, enumerated by the very same sequence 
a, (3, moreover, under the identification „4(GLB)„ ~ C(GL(n,q)) ~ ^l(GLU)„ 
unipotent traces of GLB and GLU are the same functions. 
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When we switch from GLB to GLU Borel subgroup B gets replaced by 
U C GLU which is the group of unipotent upper-triangular matrices. Note that, 
generally speaking, conjugations do not preserve U. However, we can still define 
a conjugation-invariant measure /i through the property fJ,(X) = fJ.(Y) for every 
X C U, Y C U such that X = gY g~ x . We state and prove a partial result to- 
wards the classification theorem for ergodic conjugation-invariant measures on U, 
see Conjecture 14.51 and Proposition 14.71 This theorem is a particular case of a gen- 
eral statement describing specializations of the algebra of symmetric functions A 
non-negative on Macdonald polynomials (our case corresponds to Hall-Littlewood 
polynomials), which is known as Kerov's conjecture, see |K03[ Section II. 9]. We 
also state a conjectural Law of Large numbers for ergodic measures, see Conjecture 
14.51 One particular case of this conjecture was proved by Borodin |Blj . |B2| who 
studied uniform measure on U. Finally, in Theorem l4.6l we explain which measures 
in the above conjectural classification correspond to extreme unipotent traces. 

1.4. Construction of representations of GLB. Our next topic is the construc- 
tion of representations of GLB corresponding to the extreme unipotent traces of 
„4(GLB). Of course, there exists an abstract general (Gelfand-Naimark-Segal) con- 
struction for the representation with given trace, but since by its definition GLB is 
a transformation group we seek for more explicit constructions based on its natural 
action in the infinite-dimensional vector space over ¥ q . 

In Section[5]we adopt the representation formalism of |Q2j in a modified form and 
construct unitary representations T of (GLB x GLB) (in other words, we consider 
two-sided representations) in a Hilbert space H possessing a distinguished vector 
v, which is cyclic and invariant under the action of GLB diagonally embedded into 
(GLB x GLB). v defines a spherical function x( a ) — (( a > e ) v : v ): an d viewed as a 
function on ,4.(GLB) (or GLB) this function becomes our trace. There is a simple 
link between our constructions and factor representations (semifinite, in general). 
If we consider the restriction of T on the first component of GLB x GLB, then we 
get a von Neumann factor representation. 

In the most well-studied settings of asymptotic representation theory, e.g. for 
infinite symmetric group S(oo) (see |Th64| . |VK81| . |Q3| . |Ok| ) and real infinite- 
dimensional matrix groups such as U(oo) (see |Voj . |Q1| . |Q2| ) distinguished vector 
v belongs to H and corresponding factor representation is of type However, 
in our settings, since our traces are defined only on .A(GLB) instead of the whole 
group GLB, we have to use generalized vectors (distributions) v and corresponding 
factor representations we get are of type IIx,. 

Similar situations appeared before in the investigation of at least two topics of 
classical representation theory of finite-dimensional groups. In the study of unitary 
representations of general Lie groups sometimes one is led to consider type 11^ 
factor representation, see e.g. |Pu71| . |Pu74| where characters and representations 
of simply connected solvable and more general Lie groups are studied. From the 
other side, in the theory of semisimple Lie groups generalized distinguished vectors 
show upQ in harmonic analysis, i.e. when one tries to decompose highly-reducible 
representations. As a quick example, by the well-known theorem bi-regular repre- 
sentation of a finite group G equipped with a distinguished vector v — (S-function 
at identity element of G — is the direct sum of irreducible spherical representations 



Note, however, that all the representations are of type I in this theory. 
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7r A ® (tt x )* of Gelfand pair (G x G, G) (here A goes over all irreducible representa- 
tions of G). By Peter- Weyl theorem, the same decomposition is valid for a compact 
Lie group, however, <5-function at identity is no longer a vector of L2 on the group, 
rather it is a generalized vector (distribution). 

Coming back to the construction of representation of GLB recall that extreme 
unipotent traces of .A(GLB) are parameterized by two sequences of non-negative 
reals {on} and {Pi}- We start from two distinguished simplest cases, where the 
corresponding factor representations are of type /. If a\ = 1 with other param- 
eters being zeros, then the corresponding representations is trivial 1-dimensional 
representation of GLB. On the contrary, if /3i = 1 with other parameters being 
zeros, then the corresponding von Neumann factor representation is of type Too and 
the construction is related to classical Steinberg representation of GL(n, q), see |Stj . 

Further we concentrate on the case Pi = for all i and J^. oli = 1, where we 
get type factor representations. To simplify the exposition let us stick to 
the case when only a.\ and 02 are non-zero. Our construction starts with natural 
action of GLB on the grassmanian Gr(V), which is the set of all subspaces of 
countable infinite-dimensional vector space V over ¥ q . (If more than two ctiS are 
non-zero, then Gr(V) gets replaced by an appropriate space of flags.) A well-known 
group-measure space (or crossed product) construction going back to the papers of 
F. J. Murray and J. von Neumann [MNj . [N] associates a von Neumann factor to 
a free ergodic action of a group on a space equipped with measure. Of course, the 
action of GLB on Gr(V) is not free and, thus, modifications are necessary. The 
known solution here is to use principal gruppoid of the equivalence relation spanned 
by the group action, in other words, we take the set Gr 2 (V) C Gr(V) x Gr(V) which 
is the graph of the equivalence relation. Gruppoids and construction of the associ- 
ated von Neumann algebras attracted a lot of attention in the literature, see |Mo| 
for a review, a somewhat simpler case when all the classes of equivalence relation 
are finite was first studied in |Kr) and |FM) . Gruppoids in the context of asymp- 
totic representation theory of symmetric groups first appeared in |VK81| . where 
the construction was based on the action of S(oo) on the set of words equipped 
with product measure. 

When classes (orbits) are uncountable (which is the case for the action of GLB 
in Gr(V)) the construction is more delicate, and even the definition of the correct 
measure on gruppoid becomes complicated. General solutions do exist here, see 
|Ha| . |Re) . however, in our case the situation is simplified by the fact that Gr(V) can 
be decomposed into Schubert cells, each of which is a B-orbit. This lets us to start 
from a measure on symbols of Schubert cells (here we use the Bernoulli measure, 
similarly to the constructions of |VK81| for S(oo)) and produce the measures on 
Gr(V) and gruppoid Gr 2 (V) using it. We end up with quasiinvariant measures, 
which allow us to construct usual unitary representation of GLB x GLB in the 
space of square-integrable functions on Gr 2 (V). As for the distinguished vector v, 
in the case of countable equivalence classes (as happens for S(oo)) the right choice 
is known to be the indicator function of the diagonal of gruppoid, see |VK81| . |FM| . 
In our case the diagonal has measure zero, so this choice is unappropriate. Because 
of that we have to use a generalized vector (distribution) v which is the integral 
along the diagonal (defined only for the continuous functions.) 
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One interesting aspect here is that the measure on GriV) we use is not GLB 
invariant and, moreover, one proves that there is no equivalent er-finite GLB 
invariant measure. In classics this would imply that the resulting crossed product 
gives a factor of type III, see e.g. |Krl Theorem 2.4], |SV1 Theorem 1.3.12], while 
we end up with type 11^ factor representations of GLB. An explanation here 
is the following: our trace on the operators of representation of GLB can not be 
extended to the operators of the multiplication by the function, as opposed to the 
situations related to the variations of the crossed product construction. 

Somewhat related question concerns cyclicity of the distinguished vector v. In 
general, there is no guarantee that GLB x GLB orbit of our vector is dense, thus, 
we have to consider the representation in the cyclic hull of v. In \ KM]. |Q3( 
Section 5], |V11| the cyclicity question for the representations of infinite symmetric 
group S{po) was discussed; for GLB this topic requires further investigations. 

We are currently unable to construct representations of GLB corresponding to 
other unipotent traces of .A(GLB). Even the case ai = /3, = for all i, which for 
the infinite symmetric group S(oo) corresponds to the biregular representation, is 
out of our reach at the moment. 

In papers |V11| , |V12| a new approach to the construction of spherical represen- 
tations (or finite factor representations on the different language) related to infinite 
symmetric group S(oo) was proposed. The authors hope that this approach might 
be extended to GLB. 

1.5. Biregular representation of GLB. The final object of our interest is the 
(bi-)regular representation of GLB. Since GLB admits a unique Haar measure 
Mglb (normalized by the condition ^glb(B) = 1) there is a well-defined two-sided 
representation of GLB in L2(GLB, /jglb)- This representation equipped with 
the distinguished distribution — (5-function at unit element of the group — fits into 
the formalism of generalized spherical representations and corresponds to a certain 
trace of .4 (GLB). As in the classical harmonic analysis on finite or compact groups 
we are interested into the decomposition of this representation. In Section |6] we 
describe the decomposition of the trace of bi-regular representation of GLB into a 
combination of extreme traces of _4(GLB). 

1.6. Motivations and comments. A well-known point of view is that the sym- 
metric group S(n) can be viewed as GL(n, q) over the field with one element, i.e. 
with q = 1. This agrees with similarities between the representation theory of 
GL(n,(7) and S(n), so it is natural to expect some similarities for n = oo as well. 

The infinite symmetric group S'(oo) is usually defined as the inductive limit of 
finite symmetric group, equivalently, S(oo) consists of all bijections of countable 
set, which permute only finitely many elements. A natural adaptation of this 
definition to GL(n, q) is the following. Realize GL(n, q) as a subgroup of GL(n+l, q) 
acting in the space spanned by the first n coordinate vectors and fixing n + 1st 
coordinate vector and consider the inductive limit of GL(n, q) with respect to such 
embeddings. In this way we get the infinite-dimensional group GL(oo, q). However, 
the representation theory of GL(oo, q) turns out to be not as rich as one could hope 
for. For instance, the set of extreme (indecomposable) characters of GL(oo,<?) 
is countable (see |Th72j . |Skj ) as opposed to the infinite symmetric group S(oo) 
(see |Th64| . |VK81| . |KOO| . |Ok| ) or infinite-dimensional unitary group U(oo) (see 
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[Voj . |VK82j . [Bo) . [00] . [BOj . [Pej . [GPj ) for which such sets comprise infinite- 
dimensional domains in R°°. This leads one to seek for other n = oo analogue of 
GL(n,q). 

The key idea here is to change the embeddings. A new definition is hinted 
by the notions of parabolic induction and restriction well-known in the repre- 
sentation theory of GL(n,q), see |Gr| . |Zel| . [F] . This leads to parabolic embed- 
dings i n : C(GL(n, q)) C(GL(n + l,q)) which are no longer induced by the 
group embeddings, see Section for the formal definition. The inductive limit of 
C(GL(n, q)) with respect to the embeddings i n is our main hero — algebra .A(GLB). 
A thorough analysis of the definitions leads to the realization of .A(GLB) as sub- 
algebra of the algebra of the functions on a group, that's how the group GLB first 
appears. Note that GL(oo, q) is a dense subgroup of GLB, so another point of view 
might be to consider GLB as a certain completion of discrete group GL(oo,g). 

The representation theory of GLB, indeed, turns out to be similar to that of 
S(oo). First, the classification of traces of .A(GLB) and characters of S(oo) are 
similar, sequences {cti} and {ft} appear in both. The similarity is even more 
striking when one considers unipotent extreme traces of .A(GLB). Their normalized 
versions are in one-to-one correspondence with extreme characters of S(oo) and 
both families have similar properties, e.g. multiplicativity and the same coefficients 
of decomposition into irreducibles of restrictions to .A(GLB) n (S(n)). 

The constructions of the representations corresponding to characters are also 
similar, although some distinctions do exist (e.g. the distinguished vector becomes 
a distribution for GLB). More precisely, the realization of representations of GLB 
with unipotent traces with non-zero parameters oij is related to the spaces of flags 
of subspaces, while corresponding representations of S(oo) are related to its exact 
q = 1 analogue which is the space of flags of subsets, see |VK81| . |TVj . 

The above facts lets us claim that the group GLB might be the right g-analogue 
of S(oo) and n = oo analogue of GL(n, q) in the context of the asymptotic repre- 
sentation theory. 

However, some of the similarities break down when we start considering represen- 
tations with non-zero ft. Representation of S(oo) with single non-zero parameter 
ft = 1 is the simple one-dimensional alternating representation, while the corre- 
sponding representation of GLB is an infinite-dimensional one; this is parallel to 
the difference between alternating one-dimensional representation of S(n) and cor- 
responding unipotent (principal) representation of GL(n, q) which is the Steinberg 
representation of dimension gW™ -1 )/ 2 . 

More importantly, while the (bi-)regular representation of S(oo) is irreducible 
and corresponds to zero parameters en and ft, the (bi-)regular representation of 
GLB is reducible (as we explain in Section The construction of the unipotent 
representation of GLB corresponding to zero parameters at the moment remains 
unknown. 

We intensively exploit the similarity between S(oo) and GLB in our methods. 
For instance, some theorems of the present article are based on the Ring Theorem, 
which originally was discovered in the study of S(oo), see |K V80| . |K03| and also 
|G01[ Section 8.7]). Also Schur polynomials play an important role in the study 
of S(oo), while in the present paper we intensively use both Schur polynomials and 
their q-deformation — Hall-Littlewood polynomials. 
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In the classics, the representation theory of S(oo) has numerous connections 
with the representation theory of U(oo), see |BQ2| and references therein. A q- 
deformation of the character theory of U (oo) related to the quantum groups was 
proposed in [5] . It is yet to discover whether the representation theory of GLB is 
somehow related to that g-deformation. 

Finally, we remark that some results on the structure of GLB from the algebraic 
point of view can be also found in the literature, see |Ho| . |GH| and references 
therein. 

1.7. List of main notations and theorems. 
Notations: 

S(n) symmetric group of rank n 
H q (n) Iwahori-Hecke algebra of rank n 
¥ q — finite field with q elements 

<GL(n, q) — group of all invertible n x n matrices over ¥ q 

S(oo), H g (oo), Gh(oo,q) — inductive limits of corresponding finite n objects 

GLB — group of all almost uppertriangular infinite matrices over ¥ q 

B — group of all uppertriangular infinite matrices over ¥ q 

B„ — subgroup of B of all matrices such that their top left n x n corner is 

identity matrix 

BI n — group of all uppertriangular n x n matrices over ¥ q 

GLU — group of all almost uni-uppertriangular infinite matrices over ¥ q 

U — group of all uni-uppertriangular infinite matrices over ¥ q 

Mglb — Haar measure on GLB normalized by /iGLEi(B) = 1 

Mglu — Haar measure on GLU normalized by /iGLu(U) = 1 

„4(GLB), .A(GLU) — algebra of all continuous functions with compact support 

(on the corresponding group) taking only finitely many values. 

Y — set of all Young diagrams and also Young graph 

C — set of all irreducible monic polynomials over ¥ q other than x and 1 

C n — all degree n polynomials in C 

CY n — set of all maps from from C to Y of degree n 

CY — disjoint union of sets CY„, n = 1,2,... 

Tr^j — irreducible complex representation of GL(n, q) parameterized by 

/ G CY n and its conventional character 

CY' - subset of CY of maps / such that /("a; - 1") = 0. 

A — algebra of symmetric (polynomial) functions in countably many variables 
h n , e„, p n — complete homogeneous functions, elementary symmetric functions 
and Newton power sums, respectively 
s\ — Schur function indexed by A £ ¥ 

P\(-;t), Q\(-;t) — Hall-Littlewood P and Q functions with parameter t, indexed 
by A e Y 

SPa,p,-y — homomorphism from A into C indexed by two sequence of non-negative 
numbers a = {cti}, f3 = and real number 7 such that + A) < 7> an d 

given by its values on power sums 

Sp a ,p„\pi] = 7, Sp a ,p n [Pk] = ^2^ + {-l) k - 1 J2l 3 t k > 1 
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Key theorems: 

Proposition 12.61 on page Q31 identifies _4(GLB) with the inductive limit of the 
group algebras C(GL(n, <?)). 

Theorem \2.26\ on page [19] provides the description of all extreme traces of 
.A(GLB). 

Theorem \3.3\ on page [2T] gives the proof of multiplicativity of extreme unipotent 
traces of A{GLB). 

Theorems \S.4\ and \ 3. 5\ on page |2"21 relate the values of extreme unipotent charac- 
ters to specializations of Hall-Littlewood polynomials. 

Theorem \3.10\ on page [28] identifies the restrictions of unipotent traces with 
extreme traces of Iwahori-Hecke algebra. 

Theorems \4--l\ and \4-6\ oTi pages [281 and l30l explain that each unipotent characters 
can be viewed as a probability measure. 

Conjecture \4-5\ on page |2"91 gives the (conjectural) classification and law of large 
numbers for conjugation-invariant probability measures on infinite upper-triangular 
matrices. 

Theorem \5.11\ on page l40l provides a construction for the representations of GLB 
related to grassmanian. 

Theorem 1 5. 12\ on page EH! provides a construction for the representations of GLB 
related to spaces of flags. 

Theorem \6.1\ on page [45] describes the decomposition of the biregular represen- 
tation of GLB. 

1.8. Acknowledgements. During the long period of first stage of the work on 
the subject last two authors (S.K. and A.V.) had many useful discussions with 
A. Zelevinsky, G. Olshansky. I. Bernstein. The first author (V.G) is grateful to 
G. Olshanski for drawing his attention to GLB and fruitful discussions at various 
stages of this work. V.G. was partially supported by RFBR-CNRS grants 10-01- 
93114 and 11-01-93105. A.V. was partially supported by RFBR grants 11-01-12092 
(OFI-M) and 11-01-00677. 

2. The group GLB and its Schwartz-Bruhat algebra .A(GLB) 

2.1. Basic definitions. Let F g be the finite field with q elements and let GL(n, q) 
denote the group of all invertible n x n matrices over ¥ q . For any matrix X we 
denote through XW its top left n x n corner. 

Definition 2.1. GLB is the group of all invertible almost upper-triangular matri- 
ces over W q in other words X = [Ay]°° =1 is an element of GLB if there exists n 
such that: 

(1) The n x n submatrix X^ n > is invertible, 

(2) Xij = for all i such that i > j and i > n, 

(3) Xu ^0fori>n. 

The group GLB is an inductive limit of groups GLB„, where 

GLB„ = {[X^] E GLB | Xij = if both i > j and i > n}, 

in particular, GLBo = B C GLB is the subgroup of all upper-triangular invertible 
matrices. 

Each GLB„ is a compact group (with topology of pointwise convergence of ma- 
trix elements). GLB as an inductive limit of GLB„ is a locally compact topological 
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group. Let /jglb denote the biinvariant Haar measure on GLB normalized by the 
condition /iGLB(B) = 1. 

The space Li(GLB, /iglb) is a Banach involutive algebra with multiplication 
given by the convolution. 

Definition 2.2. .A(GLB) is defined as the subalgebra of Lx(GLB, /iglb) formed 
by all locally constant functions with compact support. In other words, a function 
f(X) belongs to _4(GLB) if their exists n and a function f n : GL{n,q) — > C such 
that: 



/(*) 



fn(X^), tfXe GLB n , 
0, otherwise. 



Clearly, _4(GLB) is dense in Li (GLB, ^glb)- Note that algebra .A(GLB) does 
not have a unit element. 

Definition 2.3. A continuous function \ '■ .A(GLB) — > C is a trace o/„4(GLB) 
if 

(1) x is central, i.e. x(WU) = x(UW), 

(2) x is positive definite, i.e. x(W*W) > for any W G „4(GLB), 

Remark. It is impossible to normalize the traces, i.e. for any a G _4(GLB) 
there exists a trace x such that x( a ) = 0- 

A trace x is indecomposable if x — + a 2X2 with a% > 0, a<i > implies 

that both xi an d X2 are multiples of x- 111 other words, indecomposable traces are 
elements of extreme rays of the convex cone of all traces. 

2.2. A(GLB) as an inductive limit. For any matrix g G GL(n,q) let i^ LB G 
.A(GLB) denote the function 



I^(X) 



1, if X G GLB„ and iW = g, 
0, otherwise. 



Let e(n) denote the identity element of GL(n,t/). Then by the definition 

rGLB/ v~\ rGLB/ v n — l\ n fGLB 

Definition 2.4. .A(GLB)„ is defined as the linear span o//^ iLB ; g G GL(n, g). 
Pm£ «i otherwise, _4(GLB)„ consists of functions from „4(GLB) with support in 
GLB n and depending only on the restriction of operator g G GLB„ on V n C Voo- 

The following proposition is straightforward 

Proposition 2.5. _4(GLB)„ is a subalgebra of „4(GLB) isomorphic to the con- 
ventional group algebra C(GL(n, c/)). If e g denotes the natural basis o/C(GL(n, q)) 
then the isomorphism is given by e g — > (q — l)™q n ("- 1 )/ 2 /GLB 

Observe that „4(GLB)„ C .4(GLB)„ +1 . In the basis i^ LB this inclusion is 
given by 

■ . rGLB rGLB 
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where for g G GL(n, q) we have 

Ext GIjl3 {g) = {[hij] e GL(n + l,g) | 



= g and h n j = h n> 2 = ■■■ = /i n ,n-i = °|- 



Summarizing the discussion of this section we get the following statement. 

Proposition 2.6. The algebra .A(GLB) can be identified with the inductive limit 
of algebras _4(GLB)„: 

A(GLB) = lim „4(GLB)„ = |J^(GLB)„. 

n->oo n 

For every n the algebra _4(GLB)„ is isomorphic to the group algebra C(GL(n, q)) 
Thus, .A(GLB) is a locally semisimple algebra. 

2.3. Facts from representation theory of GL(n, q). Let us fix the notations 
and recall some basic facts from the representation theory of the group GL(n, q) 
which immediately translate into the statements for the representations and traces 
of algebra _4(GLB)„. To a large extent we adopt the notations of the book \'\\\. 

A Young diagram A is a finite collection of boxes arranged in rows with nonin- 
creasing row lengths A;. The total number of boxes in A is denoted by |A|. Let 
Y denote the set of all Young diagrams. We agree that the empty set G Y and 
|0| = 0. Y„ C Y stays for the set of all Young diagrams with n boxes. Also for the 
Young diagram A its transpose diagram is denoted A'; the row lengths of A coincide 
with column lengths of A'. For a box □ <G A its hook length h(d) is one plus number 
of the boxes below □ (in the same column) plus number of the boxes to the right 
from □ (in the same row). 

For d > 1 let Cd denote the set of all monic irreducible polynomials of degree d 
over ¥ q . Let C\ be the set of all linear polynomials x — a, a € ¥*, i.e. we exclude 
the polynomial x. Clearly, \C\\ = q — 1. Let C = U*Li ^d- 

Definition 2.7. A family of Young diagrams over the set C is a map 

4> : C Y, 

such that 

d cec d 

We call \cj>\ the degree of cj>. 

Definition 2.8. CYd is the set of all families of degree d and CY = UrfLo^-^d- 

Theorem 2.9. Irreducible representations of GL(n, q) are parameterized by el- 
ements of CYd- The dimension of irreducible representation parameterized by 
4> G CYd is given by the q-analogue of the hook formula 

n dn(4>(c)') 

dim ? (4>) = («f -l)...(g-l)n n 

d>l 



1 11 TT (a dh (b) - 1 V 
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For the proof, construction of the representations and their characters see |Gr| . 
|Zel| . [M] , For / G CYd let ir? denote the corresponding irreducible representation, 
H(irf) the space of this representation, and let x^(-) be its conventional character 
(i.e. matrix trace of %* (•)). 

Corollary 2.10. The set of all traces of „4(GLB) n is a simplicial cone spanned 
by traces x ■ In other words, if x n * s a trace of _4.(GLB)„, then there exist unique 
nonnegative coefficients c{f) such that 

x n {-) = E <f)x f (-y 

/ec¥„ 

Proof. „4(GLB)„ is isomorphic to the conventional group algebra of GL(n, q). Un- 
der this correspondence a trace of ^4(GLB)„ turns into the character of GL(n, q), 
i.e. central (class) positive-definite function on the group. It is well-known that 
characters of a finite group form a cone spanned by the characters (matrix traces) 
of the irreducible representations. □ 

Next we describe the interrelations between traces and inclusions i n . 

Embed GL(n — 1, q) x GL(1, q) into GL(n, q) as the subgroup of block diagonal 
matrices. Consider the subgroup U™ C GL(n, q) consisting of unipotent upper 
triangular matrices [tiy] such that Uij is non-zero only for j = n (and u nn = 1). 
Note that GL(n - 1, q) x GL(1, q) normalizes U%. 

Theorem 2.11. Suppose that f G CY . Let H(tt^) denote the subspace of U™- 
invariant vectors in H(tt^). And let W denote the representation GL(n — l,q) x 
GL(l,g) in this subspace. Let {fi} be all families in CY n _i for which there exist 
Hi G C\ such that 

(1) fi{x) = f(x) for x ^ yi, 

(2) The difference of the Young diagrams f{yi) \ fi{y.i) is a single box. 
Finally, let f \ fi denote the family from CY\ such that (f \ fi){yi) is the one box 
diagram. 

We have 

iff = tt^ (Ei ir^h , 

i 

Proof. See e.g. [ZHJ Chapter III]. □ 

2.4. Structure of „4(GLB). We need to introduce some notations to state an 
important corollary of Theorem 12.111 

Definition 2.12. For two families f G CY n and g G CY n _i we say that g precedes 
f and write g -<glb / if 

(1) /("a; - 1") \ g("x - 1") is one box 

(2) f (u) = g{u) for all u ^ u x - 1". 

(Here "x — 1" G C\ stays for the corresponding irreducible polynomial.) 

Theorem 2.13 (Branching rule). Letn? be the irreducible representation of algebra 
,A(GLB) rl (equivalently, of the group Gh(n,q)) parameterized by f G CY n and let 

be its conventional character (i.e. matrix trace). The restrictions of and x 
to the subalgebra yl(GLB) n _! admit the following decomposition: 

Uglb)^ ^ 
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equivalently, 

nf\ =Af® x 9 , 

U(glb)„_i y , 

where M is a zero representations of „4(GLB) n _i of dimension dim(/) — 

Remark 1. By zero representation we mean the action of ^(GLB)„_i by the 
identical zero in a vector space of arbitrary dimension. 

Remark 2. Theorem 12.131 implies, in particular, that the restriction of tt^ to 
„4(GLB)„_i is multiplicity free. This property was mentioned by various authors, 
the first proof was given by A. Zelevinski jZell Chapter III] using the Hopf algebras 
approach. Now there exist simple direct proofs of this fact, see |Go| . |AG| . 

Remark 2. As opposed to the situation with parabolic embeddings, the restric- 
tions of irreducible representations of GL(n, q) to the naturally embedded subgroup 
GL(n — l,q) are not multiplicity free, see |Th71| . |Zell Chapter III, Section 13]. 

Proof. This follows from Theorem 12.111 and we use the notations of that theorem. 
Indeed, the summation in the definition of parabolic embedding i n -\ introduces 
averaging over U™ and over GL(1, q). Therefore, the parabolic embedding translates 
into the projection on GL(1, q)-invariants in W . □ 

Now the structure of locally semisimple algebra can be encoded via its Bratteli 
diagram [Br], |VK87| . [K03j . 

Definition 2.14. _B(GLB) is a graded graph supplemented with additional num- 
bers, labels of the vertices. The set £?(GLB)„ of vertices at level n is CY n . The 
label 1(f) of the vertex f €E B(GLB)„ is the dimension of the irreducible represen- 
tation o/GL(n, q) parameterized by f, the formula for its computation is given in 
Theorem \2.9\ An edge joins vertex f and vertex g is and only if g -<glb /■ 

Theorem 12.131 implies that 

Proposition 2.15. £>(GLB) is the Bratelli diagram of algebra „4(GLB). 

For convenience of the reader we recall in our settings the general procedure for 
the reconstruction of the involutive algebra by its Bratteli diagram. 

By the well-known theorem algebra „4(GLB) n is isomorphic to the direct sum of 
matrix algebras of ranks equal to the dimensions of its irreducible representations. 
Therefore, 

(2.1) A{GLB) n = Mat(l(f),l(f)) 

/S-B(GLB)„ 

The inclusions i n : _4.(GLB)„ „4(GLB)„ + i can be reconstructed as follows. For 
every / g B(GLB) n fix the embedding 

Mat(l(g),l(g))^Mat(l(f)J(f)) 
as block-diagonal matrices. Note that he we need the inequality 

(Hglb/ 
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to be satisfied. Let i g j denote the above embedding considered as a map from the 
matrix algebra corresponding to g to the matrix algebra corresponding to / viewed 
as a subalgebra of _4(GLB)„ + i. 

Now for a = X)/gs(glb)„ m f- w ^ tn m f e A/ai(Z(/), Z(/)) in (|2.ip . we set 

i n (a)= ^igjmg- 

/(EB(GLB)„ +1 ^/ 

Algebra „4(GLB) is reconstructed as the inductive limit of „4(GLB) n . 

2.5. Some subalgebras of _4(GLB). Let B„ C GL(n, q) be the (Borel) subgroup 
of all upper-triangular matrices. We call an element 

a= c(g)e g E C(GL(n,q)) 

g£GL(n,q) 

-B n -biinvariant if c(g) = c{b\gb2) for any g £ Gh(n,q) and 61,62 £ B„. Put it 
otherwise, Byj-biinvariant element is a linear combination of characteristic functions 
of double cosets B n gB n . 

Definition 2.16. The Iwahori-Hecke algebra T~l q (n) is defined as the algebra of 
T5 n -biinvariant elements in C(GL(n, q)). 

The following proposition describes the structure of H q (n). 

Proposition 2.17. The algebra H q (n) has dimension n\ and has a linear basis 
enumerated by permutation matrices to: 

SuJ = IB~7 ^ tg ' 

1 " 9 eB„uB„ 

As an algebra W q (ji) is generated by n — 1 elements S(j,i+i) (where + 1) is 
elementary transposition permuting i and i + 1) subject to relations 

(!) s (i,i+i) 3 (j,j+i) = s (j,j+i) s (i,i+i)> > 

(2) s (fc,fc+l) s (fc+l,fc+2) s (fc,fc+l) = s (fc+l : fe+2) s (fc,fc+l) s (fe+l,fe+2)' 

( 3 ) S (fe,fe+1) = (l ~ 1 )S(fc,fc+l) + os e, 

where e is identical permutation. s e is the unit element in H q (n). 

Proof. See [J, [Bern] , □ 

Let us embed 7i q (n) into TA. q (n + 1) as a subalgebra spanned by first n — 1 out 
of n generators. 

Definition 2.18. The infinite-dimensional Iwahori-Hecke algebra H q (oo) is de- 
fined as the inductive limit ofH q (n): 

n q (oo) = lim H q (n) ={jH q (n). 

n—>oo n 

Note that T-L q (oo) is generated by countably many generators s^j+i) subject to 
the same relations as in Proposition 12. 171 

Through the identification C(GL(n, q)) ~ „4(GLB)„ we can view T-L q {n) as a 
subalgebra of _4(GLB) n . The following proposition is straightforward. 
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Proposition 2.19. The restriction of the embedding i n : ,4(GLB) n — > 
,A(GLB) n +i on the subalgebra H q (n) coincides with above embedding TL q {n) — > 
H.q(n + 1), therefore, % q (oo) C „4(GLB). H q (oo) coincides with subalgebra of 
H-biinvariant functions in „4(GLB) C Li(GLB, (Uglb)- 

We want to define yet another important subalgebra of „4(GLB). Let I-b <E 
„4(GLB) denote the indicator function of B in the realization of „4(GLB) as a 
subalgebra of Li(GLB, ^glb)- 

Definition 2.20. The unipotent subalgebra ^4(Uni) is defined as a two-sided ideal 
in „4(GLB) generated by Ib- 

Our definitions imply that H q (oo) C „4(Uni) C .4(GLB). 

We also note that the Bratteli diagram of .A(GLB) described in the previous 
section is a disjoint union of countably many copies of the Young graph with shifted 
gradings and different labels of the vertices. Therefore, „4(GLB) is the direct some 
of ideals corresponding to the connected components of its Bratteli diagram. We 
remark that „4(Uni) is precisely the component consisting of families / S CY such 
that /(it) = 0, unless u = "x — 1", see also Proposition 13.11 for a related fact. 

2.6. Classification of traces of .A(GLB). Although, we are not going to use it 
directly, but the following abstract statement holds: 

Proposition 2.21. The description of traces of a locally semisimple algebra de- 
pends solely on its Bratelli diagram without labels. In other words, if X and y 
are two locally semisimple algebras, whose Bratelli diagrams have the same sets of 
vertices and edges but, perhaps, different labels of vertices, then there is a canonical 
correspondence between their traces. 

Sketch of the proof. This follows from the identification of traces with harmonic 
functions or coherent systems on the Bratelli diagram of the algebra, see |VK87| . 
|VK90| . |K03| for more details. The key idea here is that branching of traces does 
not depend on labels, for GLB this can be seen in Theorem 12. 131 □ 

In order to state the classification theorem for traces of .A(GLB) we need to 
introduce some additional notations. 

Let / £ CY be a family of Young diagrams. We call the set 

{x e C | f{x) + 0} 

the support of / and denote it supp(f). If / and g are two families of Young 
diagrams with disjoint supports, then f + g stays for the following family: 

(f(x), x e supp(f), 
g(x), x S supp(g), 
0, otherwise. 

This operation corresponds to the parabolic induction of representations of GL(n, q) 
(see e.g. [Gr], [Zell Chapter III] and (Ml Section IV.3]) 

Let A be the algebra of symmetric functions in variables x\,X2,--- (see e.g. 
[M] for all the definitions). We intensively use various generators of this algebra, 
namely, elementary symmetric functions e„, complete symmetric functions h n and 
power sums p^ : 

k 
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We also use Schur symmetric functions s\, AeY which form a linear basis in A. 
A specialization <I> of A is an algebra homomorphism: 

$ : A -> C. 

Note that any specialization of A is uniquely defined by its values on pk- In what 
follows we write the arguments of specializations in square brackets $[•]. 

Let a = {a.i} and (3 = {fa}, i = 1,2,3... be two weakly decreasing sequences of 
non-negative real numbers such that 

oo 

(2.2) ^(a 4 +ft)<7<oo- 

i=i 

Definition 2.22. For any two sequences a and (3 of non-negative reals and number 
7 satisfying (|2.2p we define the specialization Sp a ^ n through its values on the 
generators pk of A 

5pa,/s, 7 bi] = 7, SjW» l7 [p fc ] = E a ^' + (-l)*"" 1 ^^*- 

i i 

Remark. Note that if fa = and J^, on = 7, then the specialization Sp a ^ a 
boils down to the substitution of numbers on in place of formal variables Xi. 

Definition 2.23. CY' C CY is the set of families f of Young diagrams such that 
/("x-l") = 0. 

Definition 2.24. SI(GLB) is defined as the set of triplets (a, fa /), where a = 
and (3 = {fa}, i — 1,2,3... are two weakly decreasing sequences of non-negative 
real numbers satisfying (|2.2p for 7=1 and f G CY'. 

Definition 2.25. For oj G 17 (GLB) we define a trace \ u o/<A(GLB) as follows. 
For g G GL(n, q) we have X^(-^ GLB ) = if n < \f\, otherwise, 

(2.3) X"(/ S GLB H E X /+£l(A ^ ff GLB )<W,iM, 

where £a(A) is a function from CY m taking value X in "x — 1" and taking value in 
all other points. ~^ +El ^ X \ as and above, stays for the matrix trace of the irreducible 
representation o/.4(GLB)„ (Gh(n,q) ) indexed by f -\- E\(X). 

Theorem 2.26 (Classification theorem for finite traces of „4(GLB)). The extreme 
rays of the set of traces of .A(GLB) are parameterized by elements o/fi(GLB). 
For ui = {a,faf) £ fi(GLB) the corresponding ray is R+x"(0- 

Proof. For a family / G CY' let CY^ C CY denote the set of families h G CY such 
that h(u) = f{u) for all u G C \ { u x - 1"}. 

Moreover, for a family / G CY' let T-^ denote the convex cone of traces x of 
„4(GLB) such that such that for n < \ f\ the restriction yi vanishes and 

for n > l/l in the decomposition (see Corollarv l2.10p 

U(GLB)„ h&c ^ 

c(h) = unless h G CY^\ Let T" denote the set for / being the empty family. 
We claim that for any / G CY' the convex cone is affine isomorphic to T . The 
isomorphism 

§ f : T ->• 
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is given be the following formula. If X 6 T is such that 



then 



E < h )x h+f (-)- 



heCY n nCY</) 

Theorem 12.131 implies that the branching of traces from T with respect to 
restriction on subalgebras „4(GLB)„ is the same as branching of the characters 
of symmetric groups, cf. |Saj . (K03J. Therefore, Y is isomorphic to the set of 
characters of the infinite symmetric group <S'(oo), see |VK90j . |K03| . The latter 
characters were classified by Thoma |Th64) . see also |VK81) . Thoma's theorem 
implies that the extreme rays of T are parameterized by pairs a = and 
/? = {/?,}, j = 1, 2, 3 . . . of weakly decreasing sequences of non-negative real numbers 
satisfying (|2.2j) with 7=1. The ray corresponding to a pair (a, (3) is spanned by 
the character y"'' 3 such that for g £ GL(n, q) we have 

^(C B )=E^( / f B )^,wN, 

AGY„ 

We conclude that for / £ CY 1 the extreme rays of T-^ are parameterized by pairs 
(a, /?) and are given by the formula (|2.3|) for the triplet (a, (3, f). 

It remains to prove that every extreme ray of the set of traces of _4(GLB) is an 
extreme ray of one of the sets . Indeed, let x be a trace of „4(GLB). We claim 
that there exists a unique decomposition of \ m to the sum 

x= E * (/) > * (/) er/ 

feCY' 

To prove the claim consider the restrictions x^l for which the existence 

U(GLB)„ 

and uniqueness of such decomposition is immediate. This finishes the proof. □ 

3. UNIPOTENT TRACES AND THEIR VALUES 

Recall that an irreducible representation of GL(n, q) is said to be unipotent (see 
e.g. [St], [J2|) if it contains a non-zero B„-invariant vector. (Here B„ C Gh(n,q) 
is the subgroup of upper-triangular matrices.) In the above parameterization of 
irreducible representation of GL(n, q) by the families of Young diagrams, unipotent 
representations are precisely those for which f(p) = if p ^ "x — 1". 

Proposition 3.1. Let x" , w G Sl(GLB) be an indecomposable trace o/„4(GLB). 
The following conditions are equivalent: 

(1) For every n the restriction of x u t° ^4.(GLB) n is a linear combination of 
matrix traces of irreducible unipotent representations o/GL(n, g), 

(2) Restriction of x u on H q (oo) is non-zero, 

(3) Restriction of x u on „4(Uni) is non-zero, 

(4) w = (a,0,f) with f = 0. 



Proof. Equivalence of properties (1) and (4) is a corollary of Theorem 12.261 Next, 
normalized indicator function of the Borel subgroup B„ C GL(ri, q) is a unit el- 
ement of Hq(n). In the same time in a unipotent representation of GL(n, q) it 
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acts as a projection on the set of B n -invariant vectors, while in any other repre- 
sentation it acts as zero. Therefore, the value of the matrix trace of a unipotent 
representation of GL(n, q) on this indicator function is positive, and (1) implies (2). 
Since H q (oo) C .A(Uni), the property (3) follows from (2). Finally, since .A(Uni) 
is spanned by the indicator function of B and this indicator function vanishes in 
any non-unipotent representation, every element of .A(Uni) acts as zero in any 
non-unipotent representation. Therefore, the value of a non-unipotent character of 
GL(n, q) on an element of .A(Uni) is zero and (1) follows from (3). □ 

Definition 3.2. An indecomposable trace of „4(GLB) satisfying conditions of 
Provosition \3.1\ is called unipotent. 

In this section we find a number of remarkable properties of unipotent traces 
which give a relatively simple procedure for the computation of their values on 
arbitrary elements of „4(GLB). Let us sketch all these properties together first. 
Since .A(GLB)„ is isomorphic to the group algebra of GL(n, q), the traces can 
be viewed as functions on matrices from GL(n, q) for various n. Such function 
is central, i.e. its values depend on the matrix through its Jordan normal form. 
One property of these functions is their multiplicativity which expresses the value 
on arbitrary Jordan normal forms as product of values on single block Jordan 
forms. Another property is a simple relation between values on the Jordan blocks 
with eigenvalue 1 and on Jordan blocks with arbitrary other eigenvalues. Final 
component is an expression for the values on the Jordan blocks with eigenvalue 1 
in terms of specializations of modified Hall-Littlewood polynomials. 

As for the restriction of unipotent trace on Hecke algebra H q (oo) C .4(GLB), 
in this section we identify them with extreme traces of T-L q (oo) found in |VK89j . see 
also |Mel Section 7], which also gives a formula for their values. 

3.1. Values of unipotent traces: formulations. 

Theorem 3.3 (Multiplicativity theorem for unipotent traces). Let x u be an ex- 
treme unipotent trace. For g 6 GL(n,<7) let x(d) denote the value of the restriction 
°f X" t° -4.(GLB)„ ~ C(GL(n, q)) on the element e g € C(GL(n, q)). Suppose 
that a 6 GL(n, q) and b € GL(m, q) are two matrices with coprime characteristic 
polynomials, then 

X "(«)f(t)=f(a0i), 
where a © b € GL(tt, + m, q) is the block- diagonal matrix with blocks a and b. 

Remark. A very similar multiplicativity property holds for the extreme char- 
acters of the infinite symmetric group S(oo) (see |Th64| ) and infinite-dimensional 
unitary group U(oo) (see |Vo| ) . This seems to be a general infinite-dimensional 
phenomenon. 

The values of the unipotent traces on various conjugacy classes can be computed 
in terms of specializations of symmetric functions. 

Let P M (xi, X2, ■ ■ ■ ; <Z _1 ) and Q^(xx, X2, ■ ■ ■ ; q~ 1 ) denote the Hall-Littlewood P 
and Q polynomials with parameter q -1 in variables x\, . . . labeled by a Young 
diagram /x, see [Ml Chapter III]. Let M g denote the endomorphism of the algebra 
of symmetric functions 

M, : A ->• A 
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given on the Newton power sums pk by the formula 

1 

Denote 



Pk -> —Pki k>0. 



P M = M«P M , Qn = M q Q u . 

The symmetric functions P^ and are known as modified Hall-Littlewood polyno- 
mials. 

Theorem 3.4. Let \ u be an extreme unipotent trace. For g £ GL(n, q) let x(ff) 
denote the value of the restriction of to _4(GLB)„ ~ C(GL(n, 3)) on the element 
e g £ C(GL(n, q)). Suppose that the characteristic polynomial of g is (x — l) n and 
the conjugacy class (i.e. Jordan Normal form of g) is encoded by the Young diagram 
A with n boxes. Then 

where 



x (g) = q n{x) S Pa , 0A 



i i ^ 

Now let Pl n be the endomorphism of the algebra A defined through 

Pl n :A^A, Pl„(pi)=p„i 

This is a particular case of plethysm morphism, see [M] Section 1.8]. Note that Pl n 
maps m\ to m n \. 
Now set 

SPa,/3,l : = SPa,/3,l ° PL- 

Observe that 

S Pa,p,i if] = <5Pa»,-(-0)V [/] . 
where a" = ((oi)", (a 2 ) n , ...), -(-/3)" = (-(-ft)", -(-ft)", . . . )■ 



Theorem 3.5. /n i/ie settings of Theorem \3.4\ suppose that n = km, the charac- 
teristic polynomial of g is u m , where u is an irreducible (over ¥ q ) polynomial of 
degreee k and the conjugacy class (i.e. Jordan Normal form of g) is given by the 
Young diagram A with m boxes. Then 

X(g) = q kn{X) Sp k ai ^ [Qx(-,q- k ) 

Clearly, combining Theorem 13.51 and Theorem 13.31 we get the formula for values 
of the unipotent traces on arbitrary conjugacy classes. 



3.2. Values of unipotent traces: proofs. To give a proof of Theorem 13.31 we 
need some facts about the Hopf algebra related to the representations of GL(n, q). 
We follow [23] and (M) here. 

Let D n denote the the space of central (i.e. class) complex functions on GL(n, q). 
This is a finite dimensional vector space with basis of the characteristic functions 
of the conjugacy classes of GL(n, q). The latter are parameterized by the elements 
of CY n . 

Denote 

D = ® n >oD n . 
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The vector space D is a Hopf algebra with multiplication and comultiplication given 
by the operations of parabolic induction and parabolic restriction, respectively, see 
e.g. (ZeD Section 8, Chapter III] or (Ml Section IV]. 

For a family / £ CY n let Clf G D n denote the indicator function of the conjugacy 
class in GL(n, q) corresponding to /. The definition of the multiplication in D 
implies that for disjoint fi, f^ we have 

(3.1) Cl h ■ Cl h = Cl fl+f2 

Let R n C D n denote the Z-module spanned by the characters of irreducible 
representations of GL(n, q) and R = ® n >oRn- Then D = R ®z C. R is a Hall 
subalgebra of D, moreover, R is a PSH-alebra in the terminology of |Zel| . 

Proposition 3.6. We have 

(3.2) i? = (g)i? c , D = §§D C , 

cec etc 

where (^) means the tensor product of Hopf algebras, R c is Z-module spanned by the 
characters of irreducible representations \ °f GTL(n,q) such that f(u) = unless 
u = c. R c is a Hopf subalgebra of R, elements c £ C enumerate the so-called cus- 
pidal irreducible representations of &L(n,q), which are in bijections with elements 
of C and D c = R c ®i C. Each R c is isomorphic to A, under this identification 
€ R c corresponds to s/( c ). 

Proof. See (Zefl 9.3]. □ 

Next we describe how the traces of ^4(GLB) are related to algebra D. Let p GLB 
denote a distinguished degree one element of D which is the sum of characters of 
all q — 1 irreducible representation of GL(1, q). 

Let S GLB denote the convex cone of linear functionals on D satisfying: 

€ : D -> C 

(1) £[u ■ _p GLB ] = £[it], for every u G D, 

(2) £[x] > for every n and every character \ G D n of an irreducible represen- 
tation of GL(n, g). 

Let (f) — > denote the map from the set of traces of .A(GLB) to S GLB given 
by: 

^[x / ]=c(/), 

where x S D n is a character of irreducible representation of GL(n, q) indexed by 
/ £ CY n and c(f) is the coefficient in the decomposition 



(•)= E c (f)x f (-)- 



U(GLB '" feCY n 

Comparing the definitions of the set 5 GLB and Proposition 13.61 with Theorem 
12.131 we conclude that the map </>—>■ £^ gives a bijection between the set of traces 
of _4(GLB) and S GLB . 

Moreover, note that for g € GL(rt, q) belonging to a conjugacy class / we have 
(as follows from the definitions) 

(3.3) <Ke g )=Z+[Cl f ] 

As and above we use the identification ^4(GLB)„ ~ C(GL(n, q)) here. 
Now we can prove Theorem [ 
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Proof of Theorem \3.3l Let x (which is x u f° r some ui G O(GLB)) be a unipotent 
extreme trace of A(GLB). For b G R' x ~ x the values of £ x [6] were computed in 
Theorem 12.261 In particular, since specializations Sp a ^ tl are algebra homomor- 
phisms by their definition and i?" 1-1 ~ A, we have for 61, 62 G R Ux ~ r ' 

(3.4) eM 2 ]=fW?N. 

We claim that (|3.2p actually holds for general 61, 62 G -D- To prove this claim note 
that both sides of (|3.4|1 are bilinear. Therefore, it enough to check this property for 
61,62 belonging to some linear basis of D. Let us choose the basis 6^ enumerated 
by / G CY and given by 

b f := x f = JJ x /c, 

cGC 

where S i? c is the character of the irreducible representation of <&h(\f(c)\,q) 
corresponding to the family f c defined as a unique family such that f c (c) = /(c) 
and f c (u) = for u ^ c; \^ as and above is the corresponding character of the 
irreducible representation of GL(|/|, q). It remains to observe that by the definition 
of the unipotent character 

£*[b f ] = 
unless f(u) = for any 11/ "x — 1". 

Now suppose that a G GL(n, q) and b G GL(m, g) are two matrices with coprime 
characteristic polynomials belonging to conjugacy classes parameterized by / G CY„ 
and h G CY TO) respectively. Then the families / and h are disjoint, furthermore, 
006 belongs to the conjugacy class parameterized by f + h. Therefore, using (|3.4[) , 
(|3.ip and (|3.3p we obtain 

X u (a o b) = ^ [Clf +h } = [Cl f ■ Cl h ] = [Clf]? 1 " [Ch] = X u (a)x u (b) 

□ 

Proof of Theorem \3.4\ If X M is the character of the unipotent representation of 
GL(n, q) indexed by fi, then as follows e.g. from the results of [M] Chapter IV] 

X»{g) = q nW Kll 

where A is the q~ 1 Kostka number defined as the coefficient in the decomposition 
of Schur polynomials into the sum of Hall-Littlewood polynomials 

(3.5) s ll (x 1 ,x 2 ,...) = Yl K l,\ ^ (^l, 2:2, ■ • ■ ;q~ 1 )- 

A 

Therefore, 

(3-6) X u {9) = q nW E K'^pM- 

It remains to prove that 

(3-7) Q A = £i^~V 

Indeed, the Cauchy identity for Hall-littlewood polynomials (see [M] Section III. 4]) 
yields 

(3.8) ^2 p u(xi, x 2 , . ■ ■ ; q~ 1 )Q„(y 1 ,y 2l ■ ■ ■ ; q~ l )u = JJ(1 - q~ 1 x l y J )(l - x l y ] ) 

i/£Y i.j 
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Applying the map M q with respect to the variables y\ , y 2 , . . . in the identity Q3.8P 
we arrive at 

(3.9) ^P v {xx,x 2 , ■ ■ .;q~ 1 )Q„(y 1 ,y 2 , . . .]q~ x )u = JJ(1 - x^y 1 

i/£Y i,j 

In the same time the Cauchy Identity for Schur Polynomials (see (Mj Section 1.4]) 
yields 

(3.10) ^2 s v (xi,x 2 ,. ■ .;q- 1 )s„(y 1 ,y 2 , . . . = [[(1 - x.y^y 1 
Combining ([31?]) . ([3~TP]) and ({53]) we arrive at (pTT)) . □ 



We need some preparations to prove Theorem 13.51 In order to connect the 
values of unipotent traces on unipotent and on more general conjugacy classes it 
is convenient to work not with irreducible representations of GL(n,<?), but with 
representations induced from the parabolic subgroups. 

More precisely, let fi be a Young diagram with n boxes and let fl^ denote the 
set of all flags of subspaces 

Vi C V 2 C • • • C V r 

of n-dimensional vector space F™, such that dimVfc = fXi + ■ ■ ■ + /ife. Here r is the 
number of nonempty rows in /i. GL(n,<?) naturally acts in fl^. Let denote the 
character of the corresponding representation of GL(n, q) in 0(7/^). Clearly, Vv(ff) 
is equal to the number of flags in fl^ which g fixes. 

First, we claim that if the conjugacy class of g is given by a family / £ CY„ 
such that f(p) = for all but one linear polynomial p, then the value of Vv(ff) 
does not depend on this p (but, of course, depends on the Young diagram f(p))- 
Indeed, if g\ and g 2 are two such matrices, then one can be obtained from another 
by conjugation and addition of a scalar matrix. Conjugation does not change the 
character. The addition of a scalar matrix does not change the set of invariant 
subspaces of an operator, thus, also does not change the character. 

Next suppose that n = mk and let p{x) be an irreducible polynomial of degree 
k. Suppose that the conjugacy class of g £ GL(n, q) is given by a family /, /(■) is 
equal to empty set everywhere except at p and f{p) = A. This implies that |A| = m. 
Suppose also that g' y G GL(m, q k ) (note that the number of elements in the field 
changed!) is in a conjugacy class /' such that f(p') = A for linear polynomial 
p'(x) = "x — (here y g F* fc ). Our next aim is to prove that 

(3.11) r u (9) = l^ h{9 ' v) ' if ^ = ^ foralH ' 

I 0, otherwise. 

Note that (as we have shown above) the right side of (|3.11|) . actually, does not 
depend on y. 

Conjugating the matrix, if necessary, we can assume that g is made out of k x k 
blocks. On the main diagonal all the blocks are Mat(p), where 

/0 ... - Po \ 



Mat(p) 



10 ... -pi 

... 1 - Pk - 2 

Vo ... o i - Vk . x j 
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is the companion matrix of the polynomial 

p(x) =p + piX-\ hpni 1 '" 1 + x k . 

Below the diagonal all blocks are zero. Directly above the main diagonal blocks are 
either k x k identity matrices or zeros. Identity matrices form (diagonal) groups of 
lengths (from top to bottom) Ai, A2, .... 

The matrix g' can be assumed to have a similar structure (but without any 
blocks), e.g. 

(y 1 \ 

y 1 
,00 y 
9 ~ y 1 

\0 ... y) 

More formally, g' has ys on the main diagonal, zeros everywhere below the main 
diagonal and above the second (i.e. the one on top of the main diagonal) and Is 
in the second diagonal forming groups divided by zeros. In the above example the 
length of the first group, which is Ai, equals 2. 

Now let us view the n-dimensional space F™ as {¥ k ) m . Let us identify ¥ k with 
¥ q k . The main step in proving (|3.11[) is the following lemma. 

Lemma 3.7. There exists y G ¥ q k such that if V is a ¥ q -linear subspace of (F^) m 
invariant under g, then V is a ¥ g k -linear subspace invariant under g' y and vice 
versa. 

Proof. Let h be n x n matrix made out of m k x k blocks on the diagonal, each 
block is Mat(p). If we set Q = q l with large enough t, then (g — h)Q = 0. Note 
that the matrices g and h commute, therefore (g — h)Q = g® + {— h)® . We conclude 
that V is invariant under hP . 

Now let us consider the field ¥ q \x]/p(x) ~ ¥ q k. Note that Mat(p) is the matrix 
of the operator of multiplication by x in the basis 1, x, . . . , x k ~ 1 . Since x now can be 
viewed as an element of the field with q k elements, we conclude that Mat(p) q = 
Mat{p). Therefore, it is possible to choose large t so that Mat(p)Q = Mat(p). 
Thus, hP = h and V is invariant under h. Any element of ¥ q [x]/p ~ ¥ q k is a 
polynomial in x, therefore, V is invariant under the multiplicative group of ¥ q k. 
In other words, V is a F q fc -linear subspace. Now identifying x € ¥ q [x]/p(x) with 
y G ¥ q k we see that V is invariant under the g' y . 

In the reverse direction the statement is immediate. □ 

Now (|3.11j) becomes straightforward. Indeed, the left side of (|3.11| equals the 
number of g- invariant flags. If k does not divide ^ for some i, then Lemma 13.71 
implies that there are simply no invariant flags. And if /i^ = kvi for all k, then the 
flags from fl^ are identified with flags from fl v over bigger field ¥ q k and we arrive 
at the right side of (|3.1ip . 

Proof of Theorem \3.5\ Now we can deduce the formula for the values of the unipo- 
tent traces. Decompose x u into the sum of the characters tp 1 ^: 
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In order to calculate the coefficients c M we recall the decomposition of the characters 
of the irreducible unipotent representations of GL(n, q) into the sum of We have 

(3-12) x A = E A ^^, 

where the coefficients ^ are Kostka numbers and do not depend on q (see [St] 
and also [Ml Section 1.6] and references therein). They can be defined via the 
relations in the algebra of symmetric functions A: 

A 

where m„ is the monomial symmetric function indexed by [i. We have 



AGY„ /j£Y„ \AeY„ / 



Yj S P<X,P,l[ m Ml 



Evaluating in g and using p. lip we get 



Converting back into the sum of irreducible unipotent characters we get 

X "(o) = X Sp^rMxV)- 

AeY m 

Now the application of Theorem l3.4l (with q replaced by q k ) completes the proof. □ 

3.3. Restriction of unipotent traces to Iwahori Hecke algebra. In this sec- 
tion we explain what happens when one restricts unipotent trace of .A(GLB) on 
U q {oo). 

First, we recall a classical theorem relating representations of ~H q (n) and 
GL(n, q). 

Proposition 3.8. Both irreducible representations of TL q (n) and unipotent irre- 
ducible representations of GL(n, q) are parameterized by the set Y„ of Young dia- 
grams with n boxes. The representation of T-L q (n) indexed by X coincides with the 
restriction of the representation o/GL(n, q) indexed by A on the set of H n -invariant 
vectors and on the subalgebra H q (n) C C(GL(n,q)). In particular the restriction of 
the conventional character (matrix trace) of the irreducible unipotent representation 
o/GL(n,q) onT-L q {n) is the character of the corresponding irreducible representation 
ofH q (n). 

Proof. See e.g. [CF]. □ 

The traces of infinite Hecke algebra H q (oo) were first classified in |VK89| . re- 
cently they were also studied in |Mej . From these articles the following result is 
known. 
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Proposition 3.9. Extreme traces ofH q (oo) normalized by the condition x( s e) = 1 
are enumerated by sequences a = {cti}, ft = {fti} satisfying 

«i>a 2 >■••>(), ft 1 >ft 2 > ... > 0, ^( a , +PJ < 1. 

i 

The decomposition of the restriction of trace x a '^ on H q (n) into traces \ x of irre- 
ducible representations ofH q (n) is given by the following formula: 



m q {n) 



AGY„ 

Combining Propositions 13.81 and 13.91 with Theorem 12.261 we arrive at the fol- 
lowing statement which should be viewed as an infinite-dimensional analogue of 
Proposition [ 



Theorem 3.10 (Restriction theorem for unipotent traces). The restriction of the 
extreme unipotent trace of ,A(GLB) indexed by pair of sequences a, ft on the infinite- 
dimensional Hecke algebra H g (oo) C .A(GLB) is the extreme trace ofH q (oo) in- 
dexed by the same parameters. 

4. Identification of unipotent traces with probability measures 

For any g G GL(n, q) let Cyl^ LB denote the set of all h G GLB such that 
if LB (h) = 1. We call Cyl° LB the cylindrical set of g. Clearly sets Cyl° LB span 
the cr-algebra of Borel sets on GLB. 

Theorem 4.1. Let y" be a unipotent trace o/„4(GLB). There exist a unique 
probability measure £^ LB on B C GLB, such that for any upper-triangular matrix 
g G GL(n, q) we have: 

/a i\ w/ r GLB\ GLB ffi iGLB\ 

Proof. The g-Kostka numbers x are polynomials in q^ 1 with non-negative coef- 
ficients (see e.g. [Mj Section III. 6]). Therefore, K q A > for all fx and A. Further- 
more, also Sp a ,p,i[s\] > (see |VK90| , |K03| ). Therefore, formula (|3.6|) implies 
that the values in the left side of (|4.ip are non-negative. Consequently, we can 
define the measure £?^ LB through the equation (|4.1[) . Definitions of the functions 
7^ LB and traces readily imply that we get a well-defined probability measure. 

The uniqueness follows from Theorems 13.51 and 13.31 which prove that the trace 
X u is uniquely defined by its values on unipotent classes. □ 

Remark. Analyzing the statement of Theorem 13.51 one sees that, for general g, 
the values of X^(^ 3LB ) might be negative. Thus, if we try to extended the measure 
g u to the whole group GLB, then it will no longer be a positive measure. 

The aim of this section is to analyze the properties of measures g£f LB . At this 
point it is convenient to switch from GLB to GLU. While all the proofs remain 
almost the same, but the statements for GLU look simpler and more clear. The 
reason is that unipotent upper triangular matrices have a unique eigenvalue 1, 
while general upper-triangular matrices might have up to q — 1 different eigenvalues 
and we would have to analyze the part of measure corresponding to each of them 
separately. 
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The whole theory for GLU is very much parallel to GLB. We give an overview 
here, the details can be found in the Appendix. In the same way as for GLB 
we introduce the algebra „4(GLU) of continuous functions on GLU with compact 
support taking only finitely many values. ,4(GLU) is yet again an inductive limit 
of algebras _4(GLU)„ isomorphic to C(GL(n, q)), however, the embeddings become 
different. While the classification of traces of _4(GLU) is a bit different than that 
of .A(GLB) there is still a class of unipotent traces parameterized by sequences 
a, (3. Moreover, under the identification „4(GLU)„ ~ C(GL(n, q)) ~ „4(GLB)„ 
the restriction of unipotent trace of _4(GLU) and _4(GLB) are the same functions. 
Because of that it makes no reason to distinguish between the unipotent traces of 
A(GLV) and _4(GLB). 

For GLU the group B is replaced by the subgroup U of unipotent upper- 
triangular matrices and Theorem 14.11 transforms into Theorem 14.21 (with notions 
of the indicator function ^ LU and cylindrical set Cyl g analogous to those for 
GLB) the proof of which remains the same. 

Theorem 4.2. Let x w be a unipotent trace o/„4(GLU). There exist a unique 
probability measure £>£f LU on U . such that for any upper-triangular matrix g £ 
G-L(n, q) we have: 

(4.2) x"(/ 3 GLU ) = ^ LU (Cyl^ LU ) 

The measures corresponding to unipotent traces belong to a more general class 
of measures which we now describe. 

Definition 4.3. A probability measure g on U is called central if g(Cy/^ ;LU ) de- 
pends only on the conjugacy class, i.e. on the Jordan normal form of g. In other 
words, g is invariant under conjugations by elements o/GL(oo,g). 

Remark. Conjugations, in general, do not preserve the set of upper-triangular 
matrices, so the invariance means that if both M C U and gMg -1 C U for some 
measurable set M and g £ GL(oo, q), then the measures of M and gMg' 1 are 
equal. 

Definition 4.4. A central measure g is called ergodic if it is an extreme point of 
the convex set of all central probability measures. 

The following conjecture describes the set of all ergodic central measures on U. 
For a matrix u £ U, as above, u^ n ' stays for its top-left n x n corner. Since all the 
eigenvalues of u are Is, the Jordan normal form of vS n ' can be parameterized by a 
Young diagram A, let Xi(u,n) and A-(w,n) denote the row and column lengths of 
A, respectively. 

Conjecture 4.5 (Classification and law of large numbers for ergodic central mea- 
sures). Let $ be an ergodic central measure on U. There exist two sequences rj, Ci 
(row frequencies and column frequencies), such that for every i, ^-almost surely 

\i(u,ri) A-(u,n) 
lim = r,, nm = Cj. 

n—>oo TL n— >oo TL 

Moreover, for each pair of sequences r = (r\ > r2 > ■ ■ ■ > 0) and c = {c\ > ci > 
••• > 0) satisfying X)i( r i + c «) — 1 there exists a unique ergodic central measure 
■& r ' c with corresponding row and column frequencies. 
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The $ r ' c -probabilities of cylindrical sets can be expressed through the row fre- 
quencies and column frequencies via the formula 

-n(n-l)/2 

(4.3) tf(Cyl g GLU ) = Q q^Sp^^Qx^q- 1 )} 

where g £ GL(n, q) is an unipotent upper-triangular matrix corresponding to the 
conjugacy class X, Q\ is, as above, the Hall-Littlewood polynomial and for a se- 
quence c = Ci, C2, . . . , the sequence S- q > is obtained by rearranging two-dimensional 
array of numbers (1 — q~ l )ciq l ~'J , i, j = 1,2,... in decreasing order. 

Remark 1. Conjecture 14.51 is a particular case of the conjecture on Macdon- 
ald polynomials stated in [K03, Section II. 9] and which is now known as Kerov 
conjecture. A part of this conjecture was also briefly mentioned in Section 4 of 

Remark 2. If row frequencies j*, form a geometric series (1 — q 1 ), (1 — 
q )q , ■ ■ ■ and column frequencies a are zero, then (see [HI Exercise 1 in Section 
III.2]) 

S Pr , cM AQx(-,q- 1 )] = (l-q- 1 rq- nW 

and i9 r ' c becomes the Haar (put if otherwise, uniform) measure on U. The row and 
column frequencies for the Haar measure on U were first found by A. Borodin in 

Remark 3. If — and c = (1, 0, 0, ... ) then $ r ' c is the delta-measure on the 
identity matrix. 

Remark 4. If r = (1, 0, . . . ) and Cj = then t9 r,c is the uniform measure on 
matrices u 6 U such that u — Id has maximal possible rank. In other words, all 
matrix elements of u on the second diagonal are non-zero. 

The existence of row and column frequencies for the ergodic central measure 
can, in principle, be deduced from the ergodicity. The hard (and still open) part of 
Conjecture 14.51 is the fact that the frequencies uniquely define the measure. Below 
we prove two partial results towards Conjecture I4.5[ in particular, we show that 
the measure $ r,c with cylindrical probabilities given by (|4.3j) is, indeed, an ergodic 
central measure. But, first, let us explain the relation of measures -d r ' c to the 
unipotent traces. 

Theorem 4.6. The measure £»£f LU is an ergodic central probability measure on 
U. More precisely, if uj = (a, (3), then q u = $ a 9 , where for a sequence a = 
ai,a 2 , ■ ■ ■ , the sequence is obtained by rearranging two-dimensional array of 
numbers (1 — q~ l )aiq l ~^ , i,j = l,2,... in decreasing order. 

Remark. Note the dual role of row frequencies. On one hand, the parameters 
cti , (3i of unipotent characters are limit row frequencies of Young diagrams param- 
eterizing irreducible unipotent representations of GL(n, q), see |VK81j . |KOO| . On 
the other hand frequencies show up in the limit behavior of Jordan Normal forms. 
The conceptual explanation of this double appearance of frequencies is unknown 
yet. Somehow similar phenomena is present in the asymptotic representation the- 
ory of symmetric groups with certain explanation given by the RSK algorithm, see 
[KVSfij . 
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Proof of Theorem \4-6\ Theorem 13.51 implies that the cylindrical probabilities of 
measure g w are given by. 

0a(-;? -1 )" 

Note that 

SpcpdQxhq- 1 )] = (l-q-^-MSpnMjw [Qa(-;« -1 )]- 

Comparing (j4~4|) with (|4~3|) we conclude that g u = § aiq) ^ . □ 

Now let us prove two results related to Conjecture 14.51 

Proposition 4.7. For any sequences r = {r^}, c = {c;} satisfying X)i( r i + c i) ^ 1 
the measure $ r,c with cylindrical probabilities (|4.3p is an ergodic central measure on 
U. 

Proof. The key property which we use, is the positivity of the structural constants 
of the multiplication in the basis of Hall-Littlewood polynomials. In other words, 
in the identity 

(4-5) P x (-:q- 1 )P,(-, q - 1 ) = J2^PA-,q~ 1 ) 

When q > 1 all the coefficients c\ are non-negative. This fact follows from the 
known formulas for these coefficients (see e.g. |Ra| Theorem 4.9], [Scl Theorem 
1.3] , |KM| and references therein). Since Hall-Littlewood P-polynomials and Q- 
polynomials differ by the multiplication by the constant, which is positive for q > 1 
(see [Ml Section III. 2]) we can replace P by Q in any part of (|4.5[) and the coefficients 
will be still positive. Moreover, (|4. 5[) is equivalent to the equality for skew Hall- 
Littlewood polynomials (see [Ml Section III. 5]) 

(4-6) Q„ /|t = 5>A,/^A 

A 

Again if we replace Q with P in either sides of (|4.5|) then the coefficients remain 
positive. 

Let us prove that for any sequences r,,Ci, the values 

5p r , c < 9 ),i[QA(-;9 -1 )] 

are nonnegative, which will guarantee the non- negativity of probabilities in (|4.3|) . 

First, let r = (1, 0, 0, . . . ), c = (0, 0, . . . ). Then Sp r ciq)1 P x {-; q- 1 ) = unless A is 
a one-row diagram, i.e. Ai = n, X2 = 0. In the latter case Sp r c ( q ) 1 [P\(-; q^ 1 )} = 1 ■ 
Thus, Sp r c { q ) \[Q\{'',q~ 1 )] is non-negative for all A. 

Second, let r = (0,0,...), c = (1,0,0,...). The Cauchy identity for Hall- 
Littlewood polynomials (see [Ml Section III. 4]) yields (here z and y stay for two 
sets of variables) 

/ °° 1 - q~ m 

V P x (z; q~ 1 )Q\(y; q^ 1 ) = cxp V Pm(z)p m (y) 

z — ' \ * — ' TO 

A \TO=1 



(4.4) ^(Cyl^ LU ) = q- n{n - 1)/2 q nW S Pa , 
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Applying Sp rc ( q ) x with respect to the variables y we get 

E — — - pM 

m=l m / 

= ^(l-g- 1 )™^) 

m>0 

Since P\ = e\\\ for one-column Young diagram A = (1, . . . , 1), we conclude that 
Sp r c { q ) \ [Q\(y', <Z -1 )] is zero unless A is a one-column diagram. In the latter case 
this number is positive. 

Third, if r = c = (0, 0, . . . ), then Sp r c (,j x [Q\(y; q^ 1 )} is precisely the coefficient 

of p^ in the decomposition of Q\ into the sum of products of power sums pk- 
These coefficients are known to be non-negative, see [Mj Exercise 4, Section III. 7]. 

Next, suppose that we have two specializations Spi and Sp2 of A which map Hall- 
Littlewood polynomials to non-negative numbers. Take two nonnegative numbers 
ai,a 2 and consider a new specialization Sp 1 which we call mixing of Sp\ and Sp 2l 
given by 

Sp[p k ] = [a^Sp^pk] + {a 2 ) k Sp 2 [p k ]- 
We claim that the values of Sp on Hall-Littlewood polynomials are also nonnegative. 
Indeed, this follows from the identity (see (Ml Section III. 5]) 

Sp[Qx{-\ q- 1 )] = ^(ajMSpxlQA; q-'^^-^Sp^Q^^ q' 1 )] 



and the positivity of the coefficients in (|4.6p . 

Now observe that starting with three simplest specializations which we described 
above, one can obtain any specialization Sp rc ( t ) i with finitely many non-zero r^s 
and CiS through mixing. Passing to the limit we conclude that Sp r c (,) i[Q\] is 
non- negative for all A and all sequences fj,Cj satisfying X)i( r i + c i) < 1- 

Next, let us show that the central probability measures on U are in bijections 
with linear functionals 

<P : A -> C 
satisfying three coherency properties 

(1) ^[Qx^q- 1 )] >0 for every A 

(2) 0[p!/] = 4>[f] for any / € A 

(3) 0[1] = 1. 

The correspondence is pretty much given by formula (|4.3j) and we keep the same 
notations, i.e. given a measure d the corresponding functional is 

-n(n-l)/2 

(4.7) ^(Cyl 3 GLU ) = 9 {1 _ q _ 1)n q nW MQx(-,q- 1 )}. 

The coherency properties 1. and 3. easily translate into the properties of a central 
probability measure. Let us deal with the coherency property 2. 

By the very definition, the cylindrical probabilities of measure j? should satisfy 



tf(Cyl g GLU ) = £ 0(Cyl 

heExt GLV (g) 



GLU\ 
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where 

Ext GLU {g) = <{ [h i:j ] G GL(n + 1, q) 



/i (n) = g and h nA = h na = ■■■ = h„,„_i = 0, /i n>n = l| 



is an analogue of Ext B (g) of Section^) Let us divide Ext G ™ (g) into the groups 
having the same conjugacy class. We use the formula from |B2| which says that if 
conjugacy class of g is given by the Young diagram A G Y„, then the number 
of h G Ext GlAJ (g) belonging to the conjugacy class given by the Young diagram 
/i G Y n+ i is 

= [q n q- x 'i{l-q x 'i- x U), if \ A = Oj, 
1 0, otherwise. 

Here [i \ X — Oj means that the set-theoretical difference of the Young diagrams (i 
and A is a box in column j and we agree that Ao = +oo, i.e. g A i~ A o = 0. 
Therefore, the functional defined through (|4.7|) satisfies 

— n 



(4.8) fo[Qx{-,q- 1 )]= J2 ^[Q^I^Nx^q 

t*eY n+1 q 

In the same time Pierry rules for the Hall-Littlewood polynomials (see [Ml Section 
III.5]) yield 

( uei'„+i:/i\A=n 

where j is again the column of the box \i \ X. Therefore, (|4. 8[) is equivalent to 

m [0a(-;? -1 )] =^)[p 1 QA(-;g~ 1 )]- 

Since the latter equality holds for every A and Hall-Littlewood polynomials 
Q\{'\ Q 1 ) form a linear basis of A, we arrive at the coherency property 2. 



Now we are in position to use the so-called Ring Theorem (see |KV80| . |VK90| . 
|K03| and also |GQ1[ Section 8.7]). This theorem yields, that the extreme points 
of the convex set of QA-positive functionals on A are those functional which are 
multiplicative (i.e. are algebra homomorphism) . By the very definition the func- 
tionals Sp r c ( 9 ) i are multiplicative, they also satisfy the coherency properties. We 
conclude that these functionals are extreme and, thus, the corresponding measures 
$ r ' c are indeed ergodic central measures on U. □ 

Proposition 4.8. If $ is an ergodic central measure on U, then there exist se- 
quences a, j3 satisfying (|2.2p with 7 = 1 such that the measure § has the following 
cylindrical probabilities: 

-n(rs-l)/2 

^ (Cyl G LU) = g g"WSp aA1 [Q A (.; g -l)] 



where g G GL(n, q) is an unipotent upper-triangular matrix corresponding to the 
conjugacy class X. 
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Remark. Thus, to prove that the measures i) r ' c exhaust the list of ergodic 
central measures it remains to show that if the sequence (3 is not a union of geometric 
series (with denominator q" 1 ), then Sp a ^.i [Qa(s <7~ 1 )] < f° r some A. 

Proof of Proposition \4-8\ As in the proof of Proposition l4.7l we identify ergodic cen- 
tral measures on U with multiplicative functionals on A satisfying three coherency 
properties. Observe that the coefficients of the decomposition 

s a(-) = ^cx.f.Qxi-iq" 1 ) 

are non-negative. Indeed, up to the simple constants they coincide with gr-Kostka 
numbers (see (Ml Section III. 6]). 

Therefore any multiplicative functional <j) satisfying three coherence properties 
also satisfy 

<£[sa] > 0. 

But classification of the multiplicative functionals which are non-negative on Schur 
functions is well-known. It is equivalent to the Thoma theorem on the characters 
of infinite symmetric group S(oo), see |Th64| . |VK81j , |VK90j . [K03] . The list of 
the functionals is given by 5p a ,/3,i with a,/3 satisfying (|2.2p with 7=1. □ 

5. GRUPPOID CONSTRUCTION FOR THE REPRESENTATIONS OF GLB 

In this section we give an explicit construction for the representations of GLB 
corresponding to a large class of the extreme unipotent traces of _4(GLB). 

5.1. Generalities. Generally speaking, we are going to construct irreducible gen- 
eralized spherical representations of pair (GLB x GLB, GLB). Let us introduce 
some definitions first. 

The well-known principle (see e.g. (Dj Section 13]) identifies unitary representa- 
tions of a locally-compact group G with ^-representations of Li(G) (with respect 
to Haar measure) and we will silently use this identification where it leads to no 
confusions. 

Definition 5.1. A generalized spherical representation o/(GLB x GLB, GLB) is 

a triplet: 

(1) Unitary (continuous) representation tt in a Hilbert space H with scalar 
product (■, ■): 

tt : GLB x GLB -» U(H), 

(2) Dense subspace Hi C H equipped with a norm \ ■ \, 

(3) Linear functional (distribution) v G H[ 
Satisfying the following conditions: 

(1) The inclusion i : (-Hi, | ■ |) — > (H, (■, •)) is continuous, 

(2) For any (g, h) 6 GLB x GLB, n(g, h)Hi C H x 

(3) For any a S „4(GLB) we have 7r(a, e)v G Hi and ir{e,a)v £ Hi. In other 
words, there exists w(a) such that (v, 7r(a, e)x) ~ {w(a), x) for every x £ Hi, 
and similarly for (e, a). 

(4) The span of {ir(a,b)v \ a, b G *4(GLB)} is dense in H 

(5) For any g G GLB, we have ir(g,g)v = v. 

(6) {tt(I b )v,v) = 1. 
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This definition is just a generalization of the well-known definition of a spherical 
representations of the Gelfand pair. The theory of spherical representations for the 
infinite symmetric group S(oo) and infinite-dimensional unitary group U(oo) was 
developed by G.Olshanski and his collaborators, see |Q1| . |Q2| . |Q3| . The novelty 
in the present paper is the fact that the distinguished vector v no longer belongs 
to the Hilbert space, but becomes a distribution. 

Definition 5.2. A spherical function of a generalized spherical representation 
(tt,Hi,v) is defined as 

X(a) = (n(a,b)v,v), 
where either a or b belong to „4(GLB). 

Clearly, the restriction of the spherical function to its first coordinate (i.e. to 
pairs (a, e) ) gives a trace of .A(GLB). 

The converse is also true, i.e. given a trace of ^l(GLB) we can, in principle, 
construct the corresponding representation (using a version of the so-called Gelfand- 
Naimark-Segal construction). However, the general construction is quite abstract 
and we seek for an explicit description of the representations corresponding to the 
unipotent traces \ u ■ 

We could avoid the notion of a general spherical representation and use von Neu- 
man factors instead, however, our approach seems to show more hidden structure. 
The following simple proposition explains how to pass to the factor-representations. 

Proposition 5.3. Let (tt,Hi,v) be a generalized spherical representation such that 
the corresponding traces \ °f -<4(GLB) is extreme. The restriction of tt on the 
first coordinate is von Neumann semifinite ( i. e. either type I or type II ) factor 
representation of group GLB in the cyclic span of v. 

Proof. Let V C B{H) denote the minimal von Neumann algebra containing all op- 
erators Tr(g,e), g 6 GLB. Let x' denote the (unique) extension of the trace \ 01 
_4(GLB) on V. Clearly, %' is a semifinite trace of V. Note that x' is extreme. 
Indeed, if \' = x'i + X2 > then x nas a similar decomposition and we get a contra- 
diction with extremality of x- Extremality of x' implies that V is a von Neumann 
factor. □ 

Remark. As we will see below the actual type of the factor representation can 
be different. 

5.2. Two simplest type / examples. Recall that unipotent representations are 
parameterized by two sequences a± > ct2 > ■ ■ ■ > and f3i > f3<i > • • • > such that 
^2ii a i + Pi) < 1- We start from considering some simplest cases. 

First, suppose that a\ — 1 with all other parameters being zeros. In this case 
the desired representation is just an identity representation. I.e. H is 1-dimensional 
vector space, Hi = H . tt maps all elements of GLB x GLB to identity operator 
and v is a unit vector in H . 

Next, let fi\ = 1, and let all other parameters be zeros. 

Let St n be the Steinberg representation of Gh(n,q) (see [5t], |Hu| ). This repre- 
sentation can be realized as the left representation of GL(n, q) in the right ideal of 
C(GL(n, q)) spanned by the element 

s = E t- 1 )^ E e 9 

<r€6„ a£B„ 



VADIM GORIN, SERGEI KEROV, AND ANATOLY VERSHIK 



It is well known that a linear basis of H(St n ) can be chosen to be 

{e g s | g G U n }, 

where U n is a subgroup of unipotent upper triangular matrices in GL(n, q). The 
dimension of H(St n ) is g™^" 1 )/ 2 . 

The representation St n —i of _4(GLB)„_i is naturally included into the repre- 
sentation St n of ^(GLB)„ as the subspace of [/"-invariant vectors (see Theorem 
12. Ill for the definition of the group t/"). Let St^ denote the inductive limit of the 
representations St n with respect to the above embeddings. Note that each H(St n ) 
has a (unique up to a multiplication by a constant) GL(n, (^-invariant scalar prod- 
uct and these scalar products can be choosen to agree with the above embeddings. 
Thus, the space H(St a ao ) is equipped with a scalar product. Let H(St OQ ) denote 
❖-representation of A in the completion of the space i^iSi^). 

Now let H denote the Hilbert space H{St OQ )* ® HiStoc) of Hilbert-Schmidt 
operators in H (St oo). We have a natural ^-representation ir of A x A in H. It can 
be extended to a non-degenerate representation of Li(GLB) ® Li(GLB) and, thus, 
to a unitary representation of GLB x GLB, which we denote by the same letter tt. 
Let Hi C H be the subspace of trace-class operators and let the functional v 6 H[ 
be trace. (If we identify H[ with the space B(TL) of bounded linear operators, then 
v corresponds to the identity operator.) Note that if a £ Am then the image of 
the operator <Sioo(a) lies in H(St n ). Therefore, Stoo(a) has finite rank. It follows 
that 7r((a, b))v £ Hi for any (a, b) £ A x A. All other properties are trivial and we 
conclude that (H, Hi, v) is a generalized spherical representation. One immediately 
checks that the spherical function of this representation corresponds to the trace 
with Pi = 1 and all other parameters being zeros. 

5.3. Representations related to grassmanian. We next proceed to the con- 
struction of the representation with ai = ti, ai = ti, ti+t^ = 1- Our construction 
has lots of similarities with gruppoid construction of |VK81| for the realization of 
factor representation of the infinite symmetric group S(oo). 

Let V be the infinite-dimensional linear space over ¥ q with basis ei, e%, De- 
note Vi = (ei, . . . , ei). In what follows we use an infinite-dimensional analogue of 
the well-known decomposition of grassmanian into Schubert cells. 

Definition 5.4. For a subspace X of V with di = dim(XP| V^), the symbol of X 
is the — 1 sequence di — <i;_i : 

Sym(X) := (di - d , d 2 - di, . . . ). 

Definition 5.5. For a — 1 sequence x let Schubert cell of x denote the set of all 

subspaces with symbol x: 

Sch(x) = {X C V | Sym(X) = x}. 

We fix a distinguished coordinate subspace in Sch(x) which is 

C{x) = (e i \x i = l). 

hi the same way if X is a subspace of V n , then its rt-dimensional symbol Sym n (X) 
is the 0—1 sequence (di — dg, . . . ,d n — <i n _i) of length n. For a — 1 sequence 
(xi, . . . , x n ) we define a finite Schubert cell 

Sch n (x) = {X C V n I Sym n (X) = x}. 
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By a simple linear algebra we have 

(5.1) \Sch n {x)\ = gX*=i( fa 0-™( TO + 1 )/ 2 J 

where m = yi_ j 

Remark. Another way to rewrite (|5.1[) is 

|S*c/l™(x)| =g^(- K ) ! 

where inv(— x) is the number of inversions in — x. In other words, it is the number 
of pairs i < j such that x^ < xj . Similar formula still holds when we pass from grass- 
manian to more complicated flag varieties. This makes a link to ^-exchangeability 
and GL(oo, (^-invariant measures on flags of |G02| . 

Let v x denote the uniform probability measure on the finite set Sch n (x). Thus, 
for a subspace X in V n we have 



? m(m+l)/2-E? =1 (« i ) ) X G Sch n (x), 

0, otherwise. 



For a space W (W will be either V of V n ) let Gr(W) the set of all subspaces of W . 
Note that GLB naturally acts in Gr(V). We equip Gr(V) with a topology of GLB 
space (i.e. elementary open neighborhood of a point x is the image of the action 
on x of an open neighborhood of identity element in GLB) and corresponding a- 
algebra of Borel sets. Gr(V) is a union of Schubert cells, every cell is a measurable 
subset of Gr(V) and is a B-orbit. Let ir x be the map: 

tt x : B -> Gr(V), ir{g)=gC{x). 

Let measure v x be the image of the Haar measure on B with respect to tt x . By its 
definition v x is a unique B-invariant probability measure supported on Sch(x). 
Let be the projection 

n («) : Gr(V) -> Gr(V n ), ir^(X) = Xf]V n , 

then the image of u x with respect to the map 7r(") is precisely the uniform proba- 
bility measure uV Xi x ^ on the finite Schubert cell Sch n ((xi, . . . ,x n )) C Gr(V n ). 

Let us introduce an important probability measure rj tl ^ 2 on Gr(V). Let 4> denote 
the map 

<j> : {0, 1}°° xB-> Gr{V), (x, g) -> gC{x). 

Definition 5.6. The measure i)t-t,ti is the (jj-pushforward of the product of Bernoulli 
measure with probability of 1 being t\, and Haar measure on B. In other words, 
to get a random element of Gr(V) distributed according to the measure rj tl ^ 2 we > 
first, sample a — 1 sequence x from the Bernoulli measure and then take an element 
of Sch(x) distributed according to v x . 

We also let rtf t2 be the n> n ) pushforward of rjt lt t 2 - Our definitions imply that 
for X G Gr(V n ) with symbol (xi, . . . , x n ) we have 



(5.2) v ? ut2 (x) = r (m + i)/2-j:u^)t^*% 



-J2i 



The following two propositions explain the relation between ri tl j 2 an d action of 
GLB. 
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Proposition 5.7 (Fundamental cocycle of the action on grassmanian). The mea- 
sure rjt 1 ,t 2 is quasi-invariant with respect to the action of GLB . The cocycle of the 
action of GLB is given by 

Vt lt t 2 (g • dX) = q J2 k k(Sym(X) k -Sym(gX) k ) 

ntx,t 2 (dx) 

Proof. By the definition i)tt,t 2 i s B-invariant. Thus, it remains to consider g £ 
GL(n, g) for arbitrary n. But then the computation of the cocycle of T}t lt t 2 boils 
down to the computation for 77^ t which is straightforward from (|5.2p . □ 

Proposition 5.8. Ift\ andti are nonzero, then there is no finite ora-finite GLB 
invariant measure on Gr(V) equivalent (i.e. with the same sets of measure zero) to 
Vt u t 2 - 

Proof. The classification of all finite GLB-invariant measures on Gr(V) was re- 
cently found in |GQ2| . It is shown there that every GLB-invariant probability 
measure is supported on the subspaces whose symbol has finitely many Os or Is. 
Thus, finite GLB-invariant measure cannot be equivalent to f]t u t 2 - 
For a subspace Y £ Gr{V n ) denote 

U(Y; n) = {X£ Gr(V) \Xf]V n = Y}. 

If ip is not finite, but equivalent to rit 1 .t 2 then for arbitrary k, I there exists n{k,l) 
and Z(k, I) £ Gr(V n (k,l)) such that the symbol of Z(k, I) has at least k zeros and at 
least / Is and ip(U(Z(k,l);n(k,l)) = 00. Observe that any set U(Y;n) contains an 
image of one of such sets Z(k,l) with respect to GLB-action. We conclude that 
for a non-finite GLB-invariant equivalent to rj tl .t 2 measure ip, tp(U(Y; n)) = 00 for 
any Y and n. Therefore, ip is not cr-finite. □ 

In order to get the desired spherical representation we need to introduce a more 
complicated space. The construction of this space has similarities with analogous 
construction for infinite symmetric group S(oo), see |VK81| . |TV| with some ideas 
tracing back to the papers of F. J. Murray and J. von Neumann |MN| . [N] , 

Let 

Gr 2 (V) = {(X, Y) £ Gr(V) x Gr(V) \ X = gY, for some g £ GLB}. 

We equip the set Gr 2 (V) with a topology, the elementary open neighborhoods of a 
point (X, Y) are indexed by numbers n = 0, 1, 2, . . . and 

U n (X, Y) = { (Z, W) £ Gr 2 (V) \ Z f| V n = X f| V n , W f| V n = Y f| V n , 

dim(Zf)V k ) = dim(Wf]V k ), for k > nj. 

Note that U n (X, Y) actually depends only on X nV n , Yf)V n and this set is empty 
unless dim(X n V n ) = dim(y n V n ). In this topology Gr 2 (V) is locally compact. 
The group GLB x GLB naturally acts in Gr 2 (V) and the action is continuous in 
the introduced topology. 

Now we introduce a measure pt tl t 2 on Gr 2 (V) which is quasiinvariant with re- 
spect to the action of GLB x GLB. 

Let x, y be two infinite 0—1 sequences. We write x ~ y if there exists N such that 
for n > N and £„ = i x n = E„ = i Vn- Note that (X 1 ,X 2 ) £ Gr(V) x Gr(V) 
belongs to Gr 2 {V) is and only if Sym(X 1 ) ~ Sym(X2). 
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Denote T the set of pairs (x,y) of infinite — 1 sequences such that x ~ y. T 
is equipped with sigma algebra spanned by the sets A^''"'| n , where i±, . . . ,i n and 
ji , . . . , j n are two — 1 sequences such that ^ i k = ^2 jk 

A iu'.'.'.fC = y) ^ T \ xi = ii, . . . ,x n = i n , yi = ji, . . . ,y n = j n , y k = x k , for fc > 
We define the measure Rt 1} t 2 by 

ii *l,*2( yl ii ) ...,i„J - *1 *2 

Let ^ be the map 

■0 : T x B x B — > Gr 2 (F), ^((x,y,g,h)) = (gC(x), hC(y), 
and let p tl t2 be the push-forward of the measure R tl t2 <E> /^b 55 with respect to 

The following proposition gives a more direct description of the measure pt 1 ,t 2 - 

Proposition 5.9. Let n = 0, 1, 2, . . . and (X, Y) £ Gr 2 (V). Suppose that dim(X n 
Vfc) = dim(y H Vfc) /or k > n and denote 

n 

m := dim(X n = dim(Y n V n ) = ^ Sym{X) k . 

k=l 

We have 

p t t (JJ n (X Y)) = q m ( m + 1 )^Y,k k ( S y m { X )k+Sym(Y) k )^m^n~m 

Proof. Observe that if (X', Y') are such that Sym(X') = Sym(X) and Sym(Y') = 
Sym(Y), then there exist (g, h) £ B x B such that X' = gX, Y' = hY. Therefore 

U n (X',Y') = (g,h)U n (X,Y), 

hence, by the definition of the measure, 

p tut2 (U n (X,Y))=p tut2 (U n (X',Y')). 

Now note that 

(5.3) x B x B ) = U 

(Zi,Wi) 

where Z 4 goes over q£->* HSym(X) k -m(m+i)/2 su b S paces of V such that Sym(Zi) = 
Sym(X) and Zif]V n are pairwise distinct; W goes over q I2 k HSym(Y) k - m (m+i)/2 
subspaces of V such that Sym(Wi) = Sym(Y) and Wi f] V n are pairwise distinct. 
Evaluating pt lt t 2 of both sides of (|5.3|) we get the desired formulas. □ 

Corollary 5.10. The measure pt 1 ,t 2 is quasi-invariant with respect to the action 
of GLB x GLB. The corresponding cocycle is given by 

Pti.t 2 {{g,h) ■ d(X,Yj) _ Y2 k k(Sym(X) k -Sy m (gX) k )+j: k k(Sy m (Y) k -Sym(hY) k ) 
Ptut2 {d(X,Y)) 

Proof. This follows from Proposition [O] and the fact that if (g, h) £ GLB x GLB 
and n is large enough integer, then 

(g,h)U n (X,Y) = U n (gX,hY). 

□ 
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Remark. It is easy to replace the measure pt t ,t 2 with an equivalent one pt lt t 2 
which would be invariant with respect to the action of the subgroup GLB x {e} C 
GLB x GLB. However, it is not possible to achieve the invariance with respect to 
the whole group GLB x GLB. 

Further, let 7r il:t2 denote the usual unitary representation of GLB x GLB in 
the L 2 {Gr 2 (y), Pt u t 2 )- 111 other words, for / e L 2 (Gr 2 (V), pt 1 ,t 2 ) an d (g, h) e 
GLB x GLB we have 



Let C° (Gr 2 (V)) denote the space of continuous functions on Gr 2 (V) with com- 
pact support equipped with supremum-norm. We have natural inclusions 



Consider the unitary representation of GLB x GLB dual to the restriction of T^t 1 ,t 2 
on C°(Gr 2 (V)). Somewhat abusing the notations we will use the same symbol 
7r tlit2 for this representation. 

Let Vty ti G C°(Gr 2 (V))* denote the linear functional: 



Further, let Sp(v tl: t 2 ) denote the GLB x GLB cyclic span of the linear functional 
v tlt t 2 and let Tft!,t 2 denote the restriction of ^t t ,t 2 on i2~closure of Sp(v tl-t2 ) H 



Theorem 5.11. The triplet (fct li t 2 ,C (Gr 2 (V))nSp(vt 1 .t 2 ),v tli t 2 ) is o, generalized 
spherical representation. Its spherical function gives the extreme unipotent trace of 
.A(GLB) with parameters 011 = t\, a 2 = t%- The restriction of this representations 
on the first component is von Neumann factor representation of type 11^ . 

Proof. Let us check the 6 properties of a generalized spherical representation. 

(1) The natural map i : C°(Gr 2 (V)) — > L 2 {Gr 2 {V), Pt!,t 2 ) is, indeed, a contin- 
uous inclusion. This follows from our choice of topology. 

(2) Since for any element (g, h) G GLB x GLB and any elementary open 
neighborhood U n (X, Y) (with large enough n) we have (g, h)U n (X,Y) = 
U n (gX,gY), thus, the action of GLB x GLB maps open sets to open sets 
and, therefore, preserves the space of continuous functions with compact 
support. 

(3) It suffices to prove that (I e (n) , e ) v t-i,t 2 is a continuous function with compact 
support. Let BI„ denote the subgroup Cyl^) B C B. By the definition 




C\Gr 2 {V)) C L 2 (Gr 2 (V),p tl , t2 ) C (C° (Gr 2 (V)))* . 




C°(Gr 2 (V)). 
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For X, Y £ Gr(V) let U k (X, Y) denote the indicator function of the set 
U k (X, Y). To analyze the last integral we set / = U k (I, J), k > n, I, J E 
V k and compute J geBIn /O" 1 ^, X)fi{dg). Observe that if Xf]V k ^ J 
or Xf]V„ + If) v n or dim(X f]V e ) ^ dim{If]V e ) for some n < I < 
k, then (g~ x X, X) does not belong to U k (I,J) and the integral vanishes. 
Otherwise, it is equal to 

/ f(g- 1 X 7 X) f i(dg) = [i(BI n )/M, 

where M is the number of Z E Gr(Vk) such that Z f]V n = If}V n and 
dim(Zf| V e ) = dim(If]V e ) for all n < I < k. We have 

1/M = g dim ( / )( dim ( 7 ) + 1 )/ 2 - dim ( 7n ^)( dim (- f nV'„) + l)/2-5:J =n + 1 «ym(/)f 

Therefore, the double integral equals 

(5.4) f f f(g- l X,X)n{dg)nt u t 2 {dX) 

JXeGr(V) Jg£BI n 

= ^(BI n )g dim W( dim(J ) +1)/2_dim ( InV " )( ^ t (J) 

if If]V n = Jf]V„ and dim(If)V e ) = dim(Jf|Vf) for n < £ < k, 
otherwise the double integral vanishes. (|5.4p together with formulas for 
p tl , t2 (U k (I,J)) and % fe 1>t2 (J) imply 

(5.5) f f f(g- l X,X)n{dg)nt u t 2 {dX) 



v n (X,Y)f(X,Y)p tut2 (d(X,Y)), 

<Gr 2 {V) 

where 

' ^(BI„)(|- dimtxnyotdimtxnKO+i)^^! «,m(x) ( ; if ( X}Y ) e L n , 
0, otherwise, 



(X,Y) = 



L n = {(X,Y)eGr 2 (V) 

X f) V n = Y f) V n , dim(X p| V k ) = dim(r f| V k ) for k > n}. 

Since the linear span of the functions U k (I, J) (with various k) is dense in 
L2(Gr 2 (V), pti,t 2 )> the equality (|5.5j) holds for a general function f(X,Y). 
Thus, (I e r n ), e)vt 1 .t 2 = v™ t is, as desired, a continuous function with com- 
pact support. 

(4) Since we work in the span of v tl> t 2 , there is nothing to check here. 

(5) Let us check that {g,g)vt 1 ,t 2 = v ti.t 2 f° r anv 9 G GLB. Indeed, since Trt 1 ,t 2 
is unitary representation, 

((5>5Hi,t2)/) = (vt u t 2 , (s'Ss -1 )/) 



f(gX, gxJ {9 > 9)ptl ' t2 (X, X) Vtl . t2 (dX) 

Gr(V) V Ph,t z 



Y=gX 



f(Y,Y)r, tut2 {dY) = {v tut2 ,f) 

Gr(V) 
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(6) We have already shown that I&vt 1 ,t 2 = v t ± ,t 2 - Then 
(lBV tl ,t 2 ,v tl ,t 2 ) = / Vti,t 2 {dX) = 1. 

JGr{V) 

Now we compute the trace of this representation. 
For g S Gh(n, q) we have 

(""ti, t 2 {I g , e)v tlt t2 ,v tut2 ) = (7r tl ,t 2 (5 ' 4(n),e)w tljt2 ,w tljt2 ) = (-K tlM (g,e)v n ,v) 
= M (BI n ) £ (, tl , t2 ( 5l e)g -^)(di m (x)+D/2+E t MS^(i)O w n (lj ^ 

X£Gr(V") 

By the definition for X, Y £ Gr(V n ) we have 

0, otherwise, 
where to = dim(X). It follows that 

(5.6) {TT tu t 2 (Ig,e)vt u t 2 ,v tu t 2 ) = fJ-(BI n ) ^2 if m(X) *2 

XeGr(y™):gX=X 



,dim(X) ,n- dim(X) 
t 2 



Our next aim is to decompose the function x(d) '■— i' K t 1 ,t 2 {Ig,e)'v,v) into the sum 
of matrix traces of irreducible representations of GL(n, q). We rewrite (|5.6[) as 

n 

x(g) = titr m ^m{g) 

m=0 

with 

^ m (g) = Ai (BI„)#{X g Gr(V n ) : dim(X) = m, gX = X} 

Let ^ m be the natural representation of GL(?i, q) in the space of functions on the 
set of all subspaces of V n of dimension to. If we view as a representation of 
„4(GLB)„. then its matrix trace Trace(^ m (I g ) is precisely ip m (g); the prefactor 
/i(BI„) arises from the identification of _4(GLB) and the group algebra of GL(n, q) 
(see Proposition 12. 5[) . 

The decomposition of ty m into irreducible representations is well known (see e.g. 
[St]). We have 

(5.7) * m = 0i^ ( „_ TO)tr[)iA V A , 

A 

where tp x is the irreducible unipotent representation of GL(n, q) indexed by the 
Young diagram with n boxes A and K( n _ mm }\ is the Kostka number. These num- 
bers do not depend on q and coincide with similar coefficients for the decomposition 
of the representation of symmetric group &(n) in the space of functions on the set 
of all m-element subsets of the set {1,2, ... ,n}. It is convenient for us to use yet 
another definition related to the symmetric functions: 



^ ^ ^{n— m,m), A s Ai 



where h m is the complete symmetric function and s\ is the Schur function. The 
last formula can be shown to be equivalent to the definition of Kostka numbers 
through (pn2"j) . 
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(|5.7p implies that 

x = E (E^™— ,-),A*r*r OT ) x\ 

A \ m / 

where x X is the conventional character (matrix trace) of the unipotent representa- 
tion indexed by the Young diagram with n boxes A. 
Next, observe that 

-^(n-m.rn)^™^ ™ = ^-{n-ra.m) , X m {n-m,m) (*1 1*2) = Sa(*1,*2)j 

m m<n/2 

where is the monomial symmetric function indexed by the Young diagram fi 
and for a symmetric function f(xi, x 2 , ■ ■ ■ ) the notation /(ti, £2) means the special- 
ization f(t±, t 2 , 0, 0, . . . ). 

We arrive at the final formula 

X = 5>A(ti,t 2 )x\ 

A 

which coincides with the decomposition of the extreme unipotent trace of _4(GLB) 
with parameters o>x = t\, a.% = t 2 given in Theorem 12.261 □ 

5.4. Representations related to spaces of flags. The results of Section [53] can 
be generalized to give a construction for the representations of GLB corresponding 
to the extreme unipotent representation with arbitrary sequence of parameters a,i. 
Suppose that we have r non-zero parameters ctf. 

ax = *ij a 2 = h, ■ ■ ■ , OL r = t r , 

with r being either finite or r = +00. 

Let Fl r (V) denote the space of all length 1 — 1 decreasing flags in V, i.e. 

Fl r (V) = {X x D X 2 D ■ ■ ■ D Y r _! I Xi S Gr{V)}. 

In particular, Fl 2 (V) = Gr(V). Note that, in principle, we allow non-strict inclu- 
sions in the above definition, e.g. Xi might be equal to X 2 . However, with respect 
to the measures we use, the inclusions turn out to be almost surely strict. If r = 00, 
then we also demand that |~). Xi = 0. 

The group GLB naturally acts in Fl r (V) and, similarly to the grassmanian case, 
we define: 

Fl%V) = {{F, H) e Fl r (V) x Fl r (V) \ 3g e GLB : gF = H}. 

For a flag F £ Fl r (V) let F^\ i = 1, . . . ,r — 1 denote its subspaces, i.e. F = 
D ■ ■ ■ D F^-V. The symbol Sym(F) of the flag F e Fl k (V) is defined as the 
coordinate- wise sum of the symbols of F^: 

/r-l r-l \ 

Sym(F) = Y, SymiF^Yl s V m ( F% h, ■■■)■ 

\i=l i=l / 

Note that this sum is well-defined even for r = 00 as follows from the condition 

ax l = 9. 

Let Af r denote the set {0,1..., r — 1}. For a sequence / S Af^° let Sch(f) denote 
the set of all flags in Fl r (V) with symbol /. Sch(f) has a distinguished coordinate 
flag, which we denote C(f). Sch(f) is a B-orbit and, thus, has a unique B-invariant 
probability measure. 
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Next, we define the map <fi r : 

^:JV™xB^H#), cf> r (x,g)=gC(x). 

Let r)t 1: ..^ tr be the </> r -pushforward of the product of Bernoulli measure P on Af^° 
with Prob(k) = tk and Haar measure on B. 

Let T r denote the set of pairs of sequences (x, y) € J\f£° x A/" r °° such that a: is a 
finite permutation of y. 

T r is equipped with sigma algebra spanned by the sets A 3 ^ ' '"f", where i\, . . . ,i n 
and ji, . . . , j n are two sequences from A/"" which are permutations of each other 

^u'.'.'.lin = i( X > y) e Tr | X\ = h, ■ ■ . , X n = in, VX = jx, ■ ■ ■ ,Vn = jn, Vk=X k , for k > 

Define the measure Rt u ...,t r on T r setting 

n 

"t. t ;;;;;;; i 

£=X 

Let vj) r be the map 

i> r : % X B x B -> Fl 2 r (V), <Pr{{x,y) l9l h) = (gC(x),hC(y)). 

We define the measure pt 1 ,....t T on Fl r (V) as the ?/v-pushforward of the measure 
Rt 1} ...,t r ® Mb ® Mb- 

Similarly to Proposition 15.71 and Corollary 15 . 1 01 one proves that rj tl .... : t r is GLB- 
quasiinvariant and pt lt ....t r is GLB x GLB-quasiinvariant. 

Therefore there is a natural unitary representation 7r tl tr of GLB x GLB in 
L 2 (Fl r (V), Ptu ..., tr ). 

Similarly, to Gr 2 (V) we define a topological structure of Fl r (V) and consider 
the space C°(Fl^(V)) of continuous functions with compact support. Let v tl: ...^ r £ 
C° (Flf.(V))* denote the following linear functional 

v tu ...,t r ■ C°(Fl 2 r (V)) -> C, v tlt ..., tr (f) = I f(X,X) Vtu ... ttr (dX). 

J Fl r {V) 

We further set Ttti,...,t r to be the restriction of ir^ t r on ^he L2-closure of the 
intersection of C°(Fl 2 (V)) and cyclic span Sp(v tl .... : t r ) of t r - 

Theorem 5.12. The triplet (Tr tl ,...,t r , C°(Fl*(V)) D Sp(v tl ,...,t r ), v tl ,..., tr ) is a gen- 
eralized spherical representation. Its spherical function gives the extreme unipotent 
trace of A{GLiBi) with parameters ax = t\,..., a r = t r . The restriction of this rep- 
resentations on the first component is von Neumann factor representation of type 

I loo- 

The proof repeats that of Theorem 15. f f I 

6. BlREGULAR REPRESENTATION OF GLB 

Recall that GLB is a locally compact group with biinvariant Haar measure /iglb 
and consider the Hilbert space H = i2(GLB, /xglb)- Let itReg denote the natural 
representation of (GLB x GLB) in H by left and right translations. Let Hi C H 
be the subspace C[GLB] of all continuous functions on GLB and let S e denote the 
delta- function at the identity element of GLB: 

Se(f) = f(e), fem. 
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Theorem 6.1 (On the structure of biregular representation). The triplet 
{^Regi C [GLB], 5 e ) is a generalized spherical representation of GLB. Its spher- 
ical function \ has the following decomposition into extreme traces o/.4(GLB); 

x= E c(/)x lW ' 0,/ > 

feCY> 

where l' 9 ' means the geometric series ((1 — q^ 1 ), (1 — q~ 1 )q~ 1 , (1 — q~ 1 )q~ 2 , . . . ) 
and 

dn(s(c)') 



C(/) = (g-l) l/l ] 



c6C \n n6s(c) (^ D )-i)- 

Proof. Observe that itReg{Ie(n) i e )<^e = I e (n)- Therefore, for g £ GL(n, g) we have 



nReg(Ig,e)5 e 



1,5 = e(n), 
0, otherwise. 



It follows that the restriction of x to ,A(GLB)„ (under the identification 
■A(GLB)„ ~ C(GL(n, q)) is the character of the regular representation of GL(n, q) 
multiplied by the constant 

(q - l)« g «(»-i)/2 _ ™ q _ i 

Using the well-known decomposition of the regular representation of a finite group 
into irreducibles we get 



A = IT"1 — 7 ' E dim <z( s k ff > 
U(GLB)„ f + <f - 1 t^i 



where dim 9 (/i) is the dimension of the irreducible representation of GL(n, g) indexed 
by h which can be computed using the following g-analogue of the hook formula: 

dn(s(c)') 



dim,( a ) = (q n -l)...(q-l)l 



Extracting the factor with c = "a; — 1" and using the identity (see e.g. (Ml Chapter 
I, Section 3, Exercise 2]) 

/ \ n n(X)' 

^ (l - <T\ ff- 1 ^ - 9- 1 )^- 2 (l - ■ • • J = (« 1) |A| ^ J gh( D) _ 1} 

we arrive at 
(6.1) 

= E c '(f) E ^w^fi-^.fHi-f 1 ),?- 2 !!-?- 1 ),. 



with 



A(GLB) n /eCY , ^ 



... — _- „dn{s(c)') 
C{f) = (q-l)\f\ [] rf 



It remains to compare (|6.ip with (|2.3p . □ 
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7. Appendix: GLU 

There is another distinguished infinite-dimensional group over finite field, which 
is a group of almost uni-uppertriangular matrices. 

Definition 7.1. GLU is a subgroup o/GLB defined through 

GLU = {[Xjj] G GLB : Xu = 1 for large enough i}. 

The whole theory for GLU is very much parallel to that of GLB. 

The group GLU is an inductive limit of groups GLU n : GLU = U^Lo GJLU n . 

GLU„ = {[Xij] G GLU | Xij = if both i > j and i > n; X u = 1 for i > n}, 

in particular, GLUo = U C GLU is the subgroup of all unipotent upper-triangular 
matrices. 

Each GLU„ is compact group (with topology of pointwise convergence of matrix 
elements). GLU as an inductive limit of GLU n is a locally compact topological 
group. Let /^ GLU denote the biinvariant Haar measure on GLU normalized by the 
condition /^ GLU (U) = 1. 

The space Li(GLU, /x GLU ) is a Banach involutive algebra with multiplication 
given by the convolution. 

Definition 7.2. .A(GLU) C Li(GLU, ^i GLU ) is defined as the subalgebra formed 
by all locally constant functions with compact support. In other words, a function 
f(X) belongs to .A(GLU) if their exists n and a function f n : GL(n, q) — > C such 
that: 

'fn(X^), i/IeGLU,,, 
0, otherwise. 



f(X) 



Clearly, .A(GLU) is dense in Li(GLU, ^i GLV ). Note that ^(GLU) does not 
have a unit element. 

As and above, we call a continuous function \ '■ -4(GLU) — >• C a trace of 
.A(GLU) if 

(1) x is central, i.e. x(WU) = x(UW), 

(2) x is positive definite, i.e. x(W*W) > for any W G ^l(GLU), 
Remark. It is impossible to normalize the traces, i.e. for any a G ^4(GLU) 

there exists a trace x such that x( a ) = 0- 

For any matrix g G GL(n, q) let ^ GLU G ^4(GLU) denote the function 

/GLU(X) = f 1, if X G GLU„ and = g, 
9 [ 0, otherwise. 

Let e(n) denote the identity element of &L(n,q). Then 

fGLU/ y\ _ rGLU/ Yn -1\_ „ rGLU 
1 g \ Ji )- 1 e(n) V A 9 ) ~ 9 I e(n) • 

Denote ^4(GLU)„ =< / GLU | .9 G GL(n,q) >. The following proposition is 
straightforward 

Proposition 7.3. ^4(GLU) n is a subalgebra of _4(GLU) isomorphic to the con- 
ventional group algebra C(GL(n, q)). If e g denotes the natural basis ofC(Gh(n,q)) 
then the isomorphism is given e g g™("- 1 )/ 2 /GLU 
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Observe that A(GLU) n C A(GLU) n +i- In the basis /° LU this inclusion is 
given by 

■ . tGLU tGLU 

heExt G ^ u {g) 

where for g G GL(n, q) we have 

Ext GLlJ (g) = e GL(n + l,g) | 

= g and h n .i = h n , 2 = ■ ■ ■ = h n , n -i = 0, h n . n = 

The algebra _4(GLU) can be identified with the inductive limit of algebras 
„4(GLU)„: 

A(GLU) = lim ,4(GLU)„ = |J^(GLU)„. 

n— foo n 

Thus, .A(GLU) is a locally semisimple algebra. 

Proposition 7.4. The set of all traces of A(GlAJ) n is a simplicial cone spanned 
by traces x > f £ CY n . In other words, if \ n is a trace of „4(GLU)„, then there 
exist unique nonnegative coefficients c(f) such that 

x n {-) = E <f)x f (-)- 

/eCY„ 

Proof. The proof repeats that of Proposition 12.101 □ 

Definition 7.5. For two families f G CY„ and g 6 CY„_i we say that g precedes 
f and write g -<glu / if there exists a g F* /or which 

(1) f( ll x — a")\g("x — a") is one box, 

(2) /(it) = /or m / "x — a" . 

Note that this definition is different from that of -<glb- 
Similarly to Theorem 12. 131 one proves the following statement. 

Proposition 7.6. Let it* be the irreducible representation of algebra »4(GLU)„ 
( equivalently, of the group GL(n, q) ) parameterized by f E CY n and let x^ be its 
conventional character (i.e. matrix trace). The restrictions of n-t and x^ to the 
subalgebra _4(GLU)„_i admit the following decomposition: 

x f \ = E * s > 

Uglu)^ 



equivalently, 



U(GLU)„_i 



where M is a zero representations of _4(GLU)„_i of dimension dim(/) — 
£<H GLU /dim(ff). 

We need to introduce some additional notations to state an analogue of Theorem 
CH2for GLU. 

Definition 7.7. CY' C CY is defined as the set of families f of Young diagrams 
such that f("x — a" ) = for a G F* . 
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Definition 7.8. f2(GLU) is the set of quadruples (a, /3,7, /), where 7 = {7 J } ; 
j G F*, a = {af}, P — {(3(}, i = 1, 2, 3 ... , j G ¥*; for every j the sequences a\ 
and Pf and the number r ) J satisfy (|2.2| ; moreover y\ 7 J = 1 and f G CY' . 

Theorem 7.9 (Classification theorem for finite traces of _4(GLU)). The extreme 
rays of the set of traces of A(GIAJ) are parameterized by elements offi(GLU). For 
uj = (a,/3,7, /) G fi(GLU) t/ie corresponding ray is IR + x"(-) and for g G GL(n, g) 
we /iGwe y"(-Z^ lu ) = if n< \f\, otherwise, 

(7.1) x-(/ 9 glu )= e x /+£^) ( i^) n^,^^], 

A^SYi^r |A3|=n-|/| jSF* 

Where Ej(X) G CY is a family taking value A m "x — j" (j G F*) and taking in 
aZZ other points. 

Proof. The argument starts similarly to that of Theorem l2.261 For a family / G CY' 
let CYW C CY denote the set of families h G CY such that h(u) = f{u) for all 

ueC\\J j& ,{"x-j"}. 

Moreover, for a family / G CY' let Qf denote the convex cone of traces x °f 
_4(GLU) such that such that for n < \f \ the restriction yi vanishes and 

for n > l/l in the decomposition 

w -£.**>**<•>■ 

c(h) = unless ft, G CY^l. Let denote the set 0? for / being the empty family. 
Similarly to the proof of Theorem 1 2 . 261 for any / G CY' the convex cone 0* is afhne 
isomorphic to and the statement of Theorem l7.9l is reduced to the identification 
of all extreme rays of O . The rest of the proof is this identification. 

Take q - 1 countable sets of variables (^).j =12 ,..., j G F* and let A®^ 1 ' de- 
note the algebra of (polynomial) functions symmetric in variables a^, i = 1,2, . . . 
for every fixed j G F*. Let A J denote the subalgebra of symmetric functions in 
x\,X2, For any j and any symmetric function r G A let r(x J ) denote the corre- 
sponding symmetric function in variables x^, x° 2 , ■ ■ ■ ■ In particular, pk(x J ) G A J are 
the Newton power sum in variables x J 



Clearly, the functions 



form a linear basis in A®( g ^ . 

Let A denote the cone of linear functionals w on A®' 9-1 ' satisfying: 



(1) w 

(2) w 



IljgF* s aj i 3 ?) ^ f° r an y Q ~ 1 Young diagrams {A J }. 

u (E jeF - pi(^'))1 = W M, fOT an y u e a®^- 1 ). 



FINITE TRACES AND REPRESENTATIONS OF GROUP GLB 



49 



We claim that A and 0® are affine isomorphic. Under this identification a trace 
xeS" corresponds to a functional w x such that 



n s m 



■x-j 



C(h) 



for any h G CY™ and for h G CY™ f| CY n the number c(h) is defined as the 
coefficient in the decomposition 



'U(GLU) 



= E c(/i)x h (0- 



Indeed, condition 1 translates into the non-negativity of the coefficients c(h) and 
condition 2 translates into the statement of Proposition 17.61 

Now we can again use the Ring Theorem (see |KV80| . |VK90| . |K03| and also 
|GQ1[ Section 8.7]) for studying the structure of the set A. This theorem yields 
that h G A is an element of an extreme ray (in other words, h is indecomposable) 
if and only if h = rh, where r G R+ and h G A is a multiplicative functional, i.e. 
h(uv) = h(u)h(v). 

Now let h G A be a multiplicative functional and let h? , j G F* be its restrictions 
on A- 7 '. Clearly, 



(7.2) 



n saj ( 3 



Define a new linear functional /i J on A- 7 through 

h J \u 



where deg(u) is the degree of polynomial u. The functional h J on A J is multiplicative 
and satisfies 

(1) h j [sx(x j )] > for every A G Y, 

(2) V[u P i(x j )]=W[u], 

(3) h>]p 1 (x j )] = l. 

Such functional are classified by Thoma's theorem, they correspond to extreme 
points of the set of normalized characters of S(oo), see [VK90] . |K03| . They are 
parameterized by sequences a, /3, satisfying (|2 .2[) with 7 = 1. We have: 

h^Sxix )} = Sp at p t i[sx]. 

Now set 7j = b? [pi (.t- 7 )] . Then we have 

h^sxix 3 )} = Sp a j t pj n i [s\] 
for certain a? , f3 J , 7 J . Substituting into (|7.2jl we arrive at the desired statement. 

□ 



Theorem O yields that for GLU and lu = (a, (3, f) G O(GLU) the trace x w 
is unipotent if / = and aj = f3{ = for j ^ 1. Note that if we identify 
^4(GLB)„ ~ C(Gh(n,q)) ~ i(GLU)„, then unipotent traces of .A(GLB) and 
^4(GLU) are the same functions. 
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